Modeling device, method and equipment for cable laying environment in cable trench

Abstract

The invention discloses a modeling device, method and equipment for a cable laying environment in a cable trench, and belongs to the technical field of electric power facilities. The device comprises a visible light image acquisition module used for acquiring a visible light image in a cable trench; the depth image acquisition module is used for acquiring a depth image in the cable trench; the target identification module is used for identifying pixel points in the visible light image and / or the depth image to obtain pixel points of the inner wall of the cable trench; the fusion module is used for fusing the visible light image and the depth image to obtain a fusion result; the inspection running track acquisition module is used for acquiring an inspection running track of the electronic equipment; and the model construction module is used for constructing a cable trench model according to the fusion result. According to the technical scheme, the cable trench model can be created according to the visible light image and the depth image, so that an inspector can comprehensively grasp the actual situation in the cable trench, prediction information is provided for safe operation of the cable, and dangerous factors can be found and processed in time.

Description

Technical Field

[0001] The present application belongs to the technical field of electric power facilities, and specifically relates to a device, method and equipment for modeling a cable laying environment in a cable trench. Background Art

[0002] The cable trench is an underground pipeline used for laying and replacing power or telecommunication cable facilities. It is also the enclosure structure for the laid cable facilities. There are rectangular, circular, arched and other pipeline structure forms.

[0003] At present, the inspection of cable trenches is usually carried out by staff or inspection robots entering the cable trenches to inspect the cables in the trenches. However, this method cannot grasp the overall space in the cable trench, which not only leads to the waste of cable trench and space resources, but also is not conducive to the update of stock data and the establishment of spatial attributes.

[0004] Therefore, how to construct a model of the environment in the cable trench is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a device, method and equipment for modeling the cable laying environment in a cable trench, which can create a cable trench model based on visible light images and depth images during the inspection process, so that inspection personnel can fully grasp the actual situation in the cable trench, provide predictive information for the safe operation of the cable, and facilitate timely discovery and handling of dangerous factors.

[0006] In a first aspect, an embodiment of the present application provides a device for modeling a cable laying environment in a cable trench, the device comprising:

[0007] A visible light image acquisition module, used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0008] A depth image acquisition module, used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0009] A target recognition module, used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0010] A fusion module, used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0011] A patrol operation track acquisition module is used to acquire the patrol operation track of the electronic equipment;

[0012] The model building module is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain the cable trench model.

[0013] Furthermore, the target recognition module is specifically used to:

[0014] Extracting semantic information from the visible light image, and distinguishing pixel points on the inner wall of the cable trench and pixel points on the cable according to the semantic information;

[0015] The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0016] Furthermore, the target recognition module is specifically used to:

[0017] Recognize the depth information in the depth image, and when it is recognized that the depth information has a sudden change and the sudden change meets a preset condition, recognize it as cable obstruction;

[0018] In the case of cable shielding, the pixel points shielded by the cable are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0019] Furthermore, the fusion module is also used for:

[0020] Predicting depth data of the inner wall of the cable trench of the cable shielding portion according to the depth image;

[0021] Predicting texture data of the inner wall of the cable trench of the cable shielding portion according to the visible light image;

[0022] The inner wall of the filtered portion of the cable trench is filled based on the depth data and the texture data.

[0023] Furthermore, the model building module is also used to:

[0024] When the inspection running track deviates, identifying whether the cable trench inner wall depth data changes with the running track deviation;

[0025] If the depth data changes, obtain the visible light image at the position where the depth data changes, and extract the edge information therein;

[0026] Identify and correct abnormal depth data based on the edge information

[0027] Furthermore, the device further comprises:

[0028] The cable location determination module is used to determine the distribution location of cables in the cable trench;

[0029] Accordingly, the model building module is also used for:

[0030] After the model building result is obtained, the distribution positions of the cables are marked on the cable trench model.

[0031] Furthermore, the cable position determination module is specifically used to:

[0032] Acquire an infrared image in the cable trench, and determine the cable laying position according to the infrared image and the cable working temperature;

[0033] The edge of the cable is determined according to the infrared image and the visible light image.

[0034] In a second aspect, an embodiment of the present application provides a method for modeling a cable laying environment in a cable trench, the method comprising:

[0035] Obtain visible light images in the cable trench through the RGB camera of the electronic device;

[0036] A depth image in the cable trench is obtained by a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0037] Identify pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0038] Fusing the visible light image and the depth image to obtain a fusion result of pixel points on the inner wall of the cable trench;

[0039] Obtain the inspection and operation track of electronic equipment;

[0040] According to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench, a model of the cable trench is constructed to obtain a cable trench model.

[0041] Further, pixel points in the visible light image and / or the depth image are identified to obtain pixel points on the inner wall of the cable trench, including:

[0042] Extracting semantic information from the visible light image, and distinguishing pixel points on the inner wall of the cable trench and pixel points on the cable according to the semantic information;

[0043] The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0044] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0046] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0047] In an embodiment of the present application, a visible light image acquisition module is used to acquire a visible light image in a cable trench through an RGB camera of an electronic device; a depth image acquisition module is used to acquire a depth image in a cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device; a target recognition module is used to identify pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench; a fusion module is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench; an inspection operation trajectory acquisition module is used to obtain the inspection operation trajectory of the electronic device; a model construction module is used to construct a model of the cable trench according to the inspection operation trajectory and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0048] Through the modeling device of the cable laying environment in the above-mentioned cable trench, a cable trench model can be created according to the visible light image and the depth image during the inspection process, so that the inspection personnel can fully grasp the actual situation in the cable trench, provide predictive information for the safe operation of the cable, and facilitate the timely discovery and handling of dangerous factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Example 1 of the present application;

[0050] Figure 2 It is a structural schematic diagram of a modeling device for a cable laying environment in a cable trench provided in Embodiment 2 of the present application;

[0051] Figure 3 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Embodiment 3 of the present application;

[0052] Figure 4 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Embodiment 4 of the present application;

[0053] Figure 5 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Embodiment 5 of the present application;

[0054] Figure 6 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Example 6 of the present application;

[0055] Figure 7 It is a structural schematic diagram of a device for modeling a cable laying environment in a cable trench provided in Embodiment 7 of the present application;

[0056] Figure 8 It is a flow chart of a method for modeling a cable laying environment in a cable trench provided in Embodiment 8 of the present application;

[0057] Fig. 9 It is a structural diagram of an electronic device provided in Example 9 of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0061] In the following, in combination with the accompanying drawings, the modeling device, method and equipment for the cable laying environment in the cable trench provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0062] Embodiment 1

[0063] Figure 1 Schematic diagram of the structure of the device for modeling the cable laying environment in the cable trench provided in the first embodiment of the present application. Figure 1 As shown, the specific steps include:

[0064] A visible light image acquisition module 101 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0065] A depth image acquisition module 102, used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0066] The target recognition module 103 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0067] A fusion module 104 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0068] The inspection operation track acquisition module 105 is used to acquire the inspection operation track of the electronic device;

[0069] The model building module 106 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0070] The application scenario of this solution may be a scenario in which an environmental model inside a cable trench is constructed by collecting visible light images and depth images of the cable trench.

[0071] Based on the above scenario, it can be understood that the executor of this scenario can be an intelligent terminal that constructs a cable trench model, such as a desktop computer, a laptop computer, a tablet computer, and an interactive multimedia device, etc., and no excessive limitations are made here.

[0072] The visible light image acquisition module 101 may be an RGB camera, which is arranged on an electronic device for inspecting the cable trench and is used to acquire the visible light image of the cable trench. Specifically, the RGB camera may be a camera that captures and presents images based on the three basic colors of red, green and blue. The core components of the RGB camera include a color lens and an image sensor. The working principle of the RGB camera is based on a three-color model, which assumes that all colors can be mixed from the three colors of red, green and blue.

[0073] The electronic device may be an electronic device that performs inspection tasks in a cable trench. The electronic device may move in the cable trench according to the task content or the instructions issued by the staff. The RGB camera and the depth camera installed thereon may capture the image in front of the electronic device.

[0074] A cable trench can be an underground pipeline used to lay and replace power or telecommunication cable facilities, and is also a protective structure for the laid cable facilities. There are rectangular, circular, arched and other pipeline structure forms.

[0075] Visible light images can be RGB images composed of three color spectra: red, green, and blue. Visible light images usually have good resolution and contrast, and can clearly show the outline and details of objects.

[0076] The visible light image in the cable trench can be obtained by using the color lens of the RGB camera to focus the image, while the image sensor is responsible for capturing the light signal. On the image sensor, each pixel consists of three channels, corresponding to red, green and blue light. When light is projected through the lens onto the camera's image sensor, the three channels of each pixel are measured according to the intensity of the projected light. These measurements are converted into digital signals and then processed to form a visible light image.

[0077] The depth image acquisition module 102 may be a depth camera, which is arranged on an electronic device for inspecting the cable trench, and is used to acquire a depth image in the cable trench. Specifically, the depth camera may be a camera that can sense the depth information of an object. While capturing an image, the depth camera can acquire the depth information of the object and convert it into a format required for digital signal processing and data analysis.

[0078] A depth image is a special image whose pixel values ​​represent the distance from the depth camera to each point in the scene. This image directly reflects the geometry of the visible surface of the scene.

[0079] The depth image of the cable trench can be obtained by using a depth camera to calculate the depth information of objects in the cable trench through time-of-flight technology. Specifically, the depth camera emits a beam of infrared laser or other forms of infrared light, and then measures the time it takes for these rays to be emitted and returned. By analyzing this data, the depth camera can accurately calculate the depth information of objects in the scene and convert it into a digital signal to add to the captured image.

[0080] It can be understood that in this solution, the depth camera and the RGB camera are arranged side by side in the electronic device, and in order to reduce the shooting error of the same picture, the gap between the depth camera and the RGB camera should be as small as possible.

[0081] The target recognition module 103 may be a program designed by the intelligent terminal for recognizing pixels in the visible light image and / or the depth image. Specifically, a pixel may be an image unit with a fixed position and a specific color or brightness value.

[0082] The pixel points of the inner wall of the cable trench may be pixel points that are components of the inner wall of the cable trench in the collected image data.

[0083] The method for identifying pixel points in a visible light image and / or a depth image may be to obtain a visible light image and / or a depth image of the cable, extract pixel points therein, and compare the pixel points with pixel points in a currently acquired visible light image and / or a depth image, and filter out pixel points in the currently acquired visible light image and / or a depth image that are consistent with the cable based on the comparison results, and retain the remaining pixel points as pixel points on the inner wall of the cable trench.

[0084] The fusion module 104 may be a program designed by the intelligent terminal to fuse the visible light image and the depth image. Specifically, the fusion result of the pixel points on the inner wall of the cable trench is to combine the pixel points on the inner wall of the cable trench with color or brightness value in the visible light image with the corresponding depth data to form a point cloud image of the inner wall of the cable trench containing 3D structure and color information.

[0085] The visible light image and the depth image can be fused by preprocessing the depth image, i.e., denoising, filling holes, and continuity operations to ensure the quality of the depth image. The visible light image is converted from the RGB space to the grayscale space of the depth image for easy fusion. Based on the depth information, a mapping relationship between each pixel in the visible light image and the corresponding pixel in the depth image is established. Based on the pixel correspondence, the depth information is fused into the color image to generate a 3D point cloud image of the cable trench.

[0086] The inspection operation trajectory acquisition module 105 may be a program design for the smart terminal to acquire the operation trajectory of the electronic device. Specifically, the inspection operation trajectory of the electronic device may be acquired by the smart terminal sending an operation trajectory acquisition instruction to the electronic device, and after the electronic device responds, the inspection operation trajectory of this task is packaged and sent to the smart terminal, or, after the electronic device starts the image acquisition task in the cable trench, the electronic device sends its own location to the smart terminal in real time through the positioning chip, and the smart terminal draws the location data reported by the electronic device into the inspection operation trajectory.

[0087] The model building module 106 may be a program design for the intelligent terminal to build a cable trench model according to the inspection operation track and the fusion result. Specifically, the cable trench model may be a 3D modeling of the inner wall of the cable trench, the cable rack and other actual environments to restore the scene in the cable trench.

[0088] The method of constructing the cable trench model according to the inspection operation trajectory and the fusion result can be to splice the multiple fused 3D point cloud images according to the inspection operation trajectory of the electronic equipment to obtain a complete 3D point cloud collection of the cable trench, determine the shape of the cable trench model and the positional relationship of each object in the cable trench according to the 3D structure of the point cloud collection, determine the color and material of the cable trench model according to the color information and texture information of the point cloud collection, and then complete the construction of the cable trench model.

[0089] In this solution, a program code for fusing visible light images with depth images is provided for reference:

[0090] import cv2

[0091] import numpy as np

[0092] #Load RGB image and depth image

[0093] rgb_image=cv2.imread('rgb_image.jpg')

[0094] depth_image=cv2.imread('depth_image.png',cv2.IMREAD_GRAYSCALE)

[0095] # Make sure the depth image is in the range 0-255

[0096] depth_image=np.clip(depth_image,0,255).astype(np.uint8)

[0097] #Convert the depth image to a pseudo-color image (using OpenCV's applyColorMap function here)

[0098] colored_depth=cv2.applyColorMap(depth_image,cv2.COLORMAP_JET)

[0099] #Merge the pseudo-color depth image with the RGB image

[0100] #Here we use simple alpha blending, you can also choose other methods

[0101] alpha=0.5#The alpha value controls the mixing ratio of the RGB image and the depth image

[0102] beta = (1.0-alpha)

[0103] # Make sure the RGB image and pseudo-color depth image have the same size

[0104] if rgb_image.shape[:2]! =colored_depth.shape[:2]:

[0105] rgb_image=cv2.resize(rgb_image,(colored_depth.shape[1],colored_depth.shape[0]))

[0106] #Fuse the images

[0107] fused_image=cv2.addWeighted(rgb_image,alpha,colored_depth,beta,0)

[0108] #Display or save the fused image

[0109] cv2.imshow('Fused Image',fused_image)

[0110] cv2.waitKey(0)

[0111] cv2.destroyAllWindows()

[0112] #cv2.imwrite('fused_image.jpg',fused_image)

[0113] In an embodiment of the present application, a visible light image acquisition module is used to acquire a visible light image in a cable trench through an RGB camera of an electronic device; a depth image acquisition module is used to acquire a depth image in a cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device; a target recognition module is used to identify pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench; a fusion module is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench; an inspection operation trajectory acquisition module is used to obtain the inspection operation trajectory of the electronic device; a model construction module is used to construct a model of the cable trench according to the inspection operation trajectory and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0114] Through the modeling device of the cable laying environment in the above-mentioned cable trench, a cable trench model can be created according to the visible light image and the depth image during the inspection process, so that the inspection personnel can fully grasp the actual situation in the cable trench, provide predictive information for the safe operation of the cable, and facilitate the timely discovery and handling of dangerous factors.

[0115] Embodiment 2

[0116] Figure 2 This is a schematic diagram of the structure of the modeling device for the cable laying environment in the cable trench provided in the second embodiment of the present application. This solution makes a better improvement on the above embodiment, and the specific improvement is: the target recognition module is specifically used to: extract semantic information from the visible light image, distinguish the pixel points of the inner wall of the cable trench and the cable pixel points according to the semantic information; filter out the cable pixel points to obtain the pixel points of the inner wall of the cable trench.

[0117] like Figure 2 As shown, specifically including the following:

[0118] A visible light image acquisition module 201 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0119] A depth image acquisition module 202 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0120] The target recognition module 203 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0121] A fusion module 204 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0122] The inspection operation track acquisition module 205 is used to acquire the inspection operation track of the electronic device;

[0123] The model building module 206 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0124] The target recognition module 203 is specifically used to extract semantic information from the visible light image, and distinguish the pixel points on the inner wall of the cable trench and the pixel points on the cable according to the semantic information;

[0125] The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0126] Semantic information can be obtained by analyzing and extracting the color, texture and shape of the objects expressed in the image, and then obtaining the image that is closest to human understanding. For example, by analyzing and extracting the color, texture and shape in the image, we know that there is sand, blue sky, sea water, etc. in the image, and then we get the semantics of the image as a beach.

[0127] The method of extracting semantic information from a visible light image can be to extract image features (for example, color, texture, etc.) at each position in the visible light image, learn the image features at each position, classify the areas of the visible light image according to similar image features, and determine the semantic information corresponding to each area in the visible light image based on the image features of objects such as the inner wall of the cable trench, cables, and cable racks.

[0128] To obtain the pixel points of the inner wall of the cable trench, after determining the semantic information corresponding to each area in the visible light image, the area in the image with the semantic information of the cable is deleted, and the remaining part is used as the pixel points of the inner wall of the cable trench.

[0129] The advantage of this setting of the present scheme is that the cable part in the visible light image can be identified by extracting semantic information, which is conducive to accurately filtering out the cable pixels, so as to avoid deleting the pixels on the inner wall of the cable trench, resulting in the visible light image not being able to accurately reflect the actual situation in the cable trench.

[0130] Embodiment 3

[0131] Figure 3 It is a structural schematic diagram of the modeling device for the cable laying environment in the cable trench provided in the third embodiment of the present application. This solution makes a better improvement on the above embodiment, and the specific improvement is as follows: the target recognition module is specifically used to: recognize the depth information in the depth image, and when it is recognized that the depth information has a mutation, and the mutation meets the preset conditions, it is recognized as cable occlusion; in the case of cable occlusion, the pixel points of the cable occlusion are filtered out to obtain the pixel points of the inner wall of the cable trench.

[0132] like Figure 3 As shown, specifically including the following:

[0133] A visible light image acquisition module 301 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0134] A depth image acquisition module 302 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0135] The target recognition module 303 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0136] A fusion module 304 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0137] The inspection operation track acquisition module 305 is used to acquire the inspection operation track of the electronic device;

[0138] The model building module 306 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0139] The target recognition module 303 is specifically used to recognize the depth information in the depth image, and when it is recognized that the depth information has a sudden change and the sudden change meets a preset condition, it is recognized as cable obstruction;

[0140] In the case of cable shielding, the pixel points shielded by the cable are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0141] The preset condition may be to record the depth change of the location of the cable that blocks the inner wall of the cable trench in advance, use the depth change as the preset condition, and determine whether there is cable obstruction based on whether the sudden change of the depth information currently collected is consistent with the preset condition.

[0142] The method for identifying the depth information in the depth image can be to obtain the depth information of the inner wall of the cable trench in the depth image, and predict the correlation between the depth information and the height of the inner wall of the cable trench according to the changing trend of the depth information of the unobstructed part of the inner wall of the cable trench. When the depth information does not conform to the correlation, the depth change of the inconsistent part is compared with the preset conditions. If they are consistent, it is determined that there is a cable obstruction at the current position of the inner wall of the cable trench.

[0143] The method for filtering out the pixel points blocked by the cable is that after determining that there is cable blocking, the pixel points and corresponding depth information of the part that are consistent with the preset conditions are deleted, and the other parts of the inner wall of the cable trench that violate the blocking are retained as the pixel points of the inner wall of the cable trench.

[0144] The advantage of this setting of the scheme is that the cable obstructed part is filtered out according to the variation law of the depth information of the inner wall of the cable trench, so as to retain the pixel points of the inner wall of the cable trench and avoid the cable affecting the staff's analysis and interpretation of the cable trench model.

[0145] Embodiment 4

[0146] Figure 4 This is a schematic diagram of the structure of the modeling device for the cable laying environment in the cable trench provided in the fourth embodiment of the present application. This solution makes a better improvement on the above embodiment, specifically: the fusion module is also used to: predict the depth data of the inner wall of the cable trench of the cable-shielded part according to the depth image; predict the texture data of the inner wall of the cable trench of the cable-shielded part according to the visible light image; fill the filtered part of the inner wall of the cable trench based on the depth data and the texture data.

[0147] like Figure 4 As shown, specifically including the following:

[0148] A visible light image acquisition module 401 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0149] A depth image acquisition module 402 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0150] The target recognition module 403 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0151] A fusion module 404 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0152] The inspection operation track acquisition module 405 is used to acquire the inspection operation track of the electronic device;

[0153] The model building module 406 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0154] The fusion module 404 is further used to predict the depth data of the inner wall of the cable trench of the cable shielding part according to the depth image;

[0155] Predicting texture data of the inner wall of the cable trench of the cable shielding portion according to the visible light image;

[0156] The inner wall of the filtered portion of the cable trench is filled based on the depth data and the texture data.

[0157] A method for predicting the depth data of the inner wall of the cable trench where the cable is blocked can be that the intelligent terminal analyzes and learns the changing trend of the depth information of the inner wall of the cable trench after filtering out the cable obstruction as the cable trench height changes, determines the changing function of the depth information of the inner wall of the cable trench, obtains the height of the cable blocking position, brings the height into the changing function of the depth information, and then obtains the depth information of the inner wall of the cable trench at the current height.

[0158] A method for predicting the texture data of the inner wall of the cable trench in the portion blocked by the cables may be to extract image features of the inner wall of the cable trench in the portion not blocked by the cables in the visible light image, determine the texture data of the inner wall of the cable trench based on the extracted image features, and use the texture data as the texture data of the inner wall of the cable trench in the portion blocked by the cables.

[0159] The method for filling the filtered part of the inner wall of the cable trench can be to determine the reference position of the inner wall of the cable trench of the cable shielding part according to the predicted depth data of the inner wall of the cable trench, make pixel points of the inner wall of the cable trench of the cable shielding part according to the predicted texture data, and fill the made pixel points to the predetermined reference position, so that the filled part is naturally connected with the original inner wall of the cable trench, so as to obtain a complete cable trench inner wall fusion result.

[0160] The advantage of this arrangement of the present scheme is that the inner wall of the cable trench that shields the cables can be filled to obtain a complete inner wall model of the cable trench, thereby making the constructed cable trench model more in line with the actual situation, which is beneficial for the staff to obtain complete information on the cable trench environment.

[0161] Embodiment 5

[0162] Figure 5 This is a schematic diagram of the structure of the modeling device for the cable laying environment in the cable trench provided in the fifth embodiment of the present application. This solution makes a better improvement on the above embodiment, specifically: the model building module is also used to: when there is a deviation in the inspection running track, identify whether the depth data of the inner wall of the cable trench changes with the deviation of the running track; if the depth data changes, obtain the visible light image of the depth data change position and extract the edge information therein; identify and correct the abnormal depth data according to the edge information.

[0163] like Figure 5 As shown, specifically including the following:

[0164] A visible light image acquisition module 501 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0165] A depth image acquisition module 502 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0166] The target recognition module 503 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0167] A fusion module 504 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0168] The inspection operation track acquisition module 505 is used to acquire the inspection operation track of the electronic device;

[0169] The model building module 506 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0170] The model building module 506 is further used to identify whether the cable trench inner wall depth data changes with the deviation of the running track when the inspection running track deviates;

[0171] If the depth data changes, obtain the visible light image at the position where the depth data changes, and extract the edge information therein;

[0172] Abnormal depth data is identified and corrected based on the edge information.

[0173] A method for identifying whether the depth data of the inner wall of the cable trench changes with the deviation of the motion trajectory can be to identify the depth data of the inner wall of the cable trench when it is identified that the motion trajectory moves in the horizontal direction. If the depth data at most positions change when the motion trajectory moves in the horizontal direction, and the change amplitudes of the depth data at each position are similar, it is determined that the depth data of the inner wall of the cable trench changes with the deviation of the motion trajectory.

[0174] The method of obtaining the visible light image of the position where the depth data changes can be to extract the visible light image when the motion trajectory moves horizontally after determining that the depth data of the inner wall of the cable trench changes with the deviation of the motion trajectory, and obtain the corresponding position of the position where the depth data changes in the visible light image based on the corresponding relationship between the visible light image and the depth image at the same time. In the corresponding position of the visible light image, edge detection algorithms (such as Canny, Sobel, etc.) are used to extract edge information. These algorithms can identify areas in the image where the brightness changes significantly, that is, the edges of objects.

[0175] According to the edge information, the method of identifying and correcting abnormal depth data can be to convert the extracted edge information into a binary image or mask for subsequent comparison with the depth image. In the depth image, the depth data is represented in the form of grayscale values, where the size of the grayscale value represents the distance from the object to the camera. Observe the depth image and analyze whether there are grayscale value areas that do not match the actual situation, which may contain erroneous depth data. Spatially align the edge information (binary image or mask) of the visible light image with the depth image to ensure that the two can correspond one to one at the pixel level. Compare the edge information and depth information pixel by pixel. In theory, the pixel where the edge information is located should correspond to the area where the grayscale value changes significantly in the depth image (i.e., the edge of the object). If the pixel where the edge information is located does not change significantly in the depth image or the grayscale value does not match the actual distance, it may mean that the depth data of the area is wrong. According to the comparison results of the edge and depth information, the erroneous depth data area in the depth image is identified. Interpolation calculation is performed using the surrounding correct depth data values ​​to estimate and replace the erroneous depth data values. Then the correction of abnormal depth data is completed.

[0176] The advantage of this arrangement of the present scheme is that when there is a deviation in the inspection operation trajectory of the electronic equipment, the abnormal depth data of the inner wall of the cable trench can be identified and corrected in time to avoid affecting the accuracy of the construction of the cable trench model.

[0177] Embodiment 6

[0178] Figure 6This is a schematic diagram of the structure of the modeling device for the cable laying environment in the cable trench provided in Example 6 of the present application. This solution makes a better improvement on the above-mentioned embodiment, and the specific improvement is: the device also includes: a cable position determination module, which is used to determine the distribution position of the cables in the cable trench; correspondingly, the model construction module is also used to: after obtaining the model construction result, mark the distribution position of the cables on the cable trench model.

[0179] like Figure 6 As shown, specifically including the following:

[0180] A visible light image acquisition module 601 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0181] A depth image acquisition module 602 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0182] The target recognition module 603 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0183] A fusion module 604 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0184] The inspection operation track acquisition module 605 is used to acquire the inspection operation track of the electronic device;

[0185] A model building module 606 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model;

[0186] The cable position determination module 607 is used to determine the distribution position of the cables in the cable trench.

[0187] The model building module 606 is further used to mark the distribution positions of the cables on the cable trench model after obtaining the model building results.

[0188] The cable position determination module 607 may be a program design for determining the location of the laid cables in the cable trench. The specific method for determining the distribution location of the cables in the cable trench may be to perform semantic recognition and edge extraction on the visible light image, thereby determining the area where the pixel points of the cable are located, and recording the position coordinates of the area in the image.

[0189] The distribution position of the cable can be marked on the cable trench model in that the intelligent terminal obtains the 3D model of the cable through big data, determines the laying position of the cable in the constructed cable trench model according to the distribution position of the cable obtained by the cable position determination module, and places the 3D model of the cable in the determined laying position according to the proportion of the cable trench model to obtain the cable trench model containing the cable distribution.

[0190] The advantage of this arrangement is that the distribution of cables can also be displayed in the cable trench model, which helps staff analyze the specific environment of the cable location and accurately predict the risks that may exist during cable operation.

[0191] Embodiment 7

[0192] Figure 7 This is a schematic diagram of the structure of the modeling device for the cable laying environment in the cable trench provided in Example 7 of the present application. This solution makes a better improvement on the above-mentioned embodiment, and the specific improvement is as follows: the cable position determination module is specifically used to: obtain an infrared image in the cable trench, determine the cable laying position according to the infrared image and the cable working temperature; determine the cable edge according to the infrared image and the visible light image.

[0193] like Figure 7 As shown, specifically including the following:

[0194] A visible light image acquisition module 701 is used to acquire a visible light image in the cable trench through an RGB camera of an electronic device;

[0195] A depth image acquisition module 702 is used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0196] The target recognition module 703 is used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0197] A fusion module 704 is used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0198] The inspection operation track acquisition module 705 is used to acquire the inspection operation track of the electronic device;

[0199] A model building module 706 is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model;

[0200] The cable position determination module 707 is used to determine the distribution position of the cables in the cable trench.

[0201] The cable position determination module 707 is specifically used to obtain an infrared image in the cable trench, and determine the cable laying position according to the infrared image and the cable working temperature;

[0202] The edge of the cable is determined according to the infrared image and the visible light image.

[0203] Infrared images can be images obtained through infrared radiation (light with a wavelength longer than visible light). Infrared radiation is generated by the heat emitted by objects, and this radiation can penetrate some objects or materials that visible light cannot penetrate. Therefore, infrared imaging technology is particularly suitable for applications such as night observation, target tracking, temperature measurement, and detection of hidden objects.

[0204] When obtaining an infrared image in a cable trench, an infrared camera can be used to photograph the cable trench. The infrared camera works based on the infrared light emitted or reflected by the target object. Specifically, the infrared camera collects the infrared light emitted by the object in the cable trench through the lens and focuses it on the infrared sensor. Then, the sensor converts the infrared light into an electrical signal, which is processed by the image processor and output as a visible image. In order to reduce the positioning error between the infrared image and the visible light image, the infrared camera and the RGB camera are set side by side at a specified position of the electronic device.

[0205] The method for determining the cable layout position can be to obtain the average temperature of the cable during normal operation, obtain the area that meets the cable operating temperature in the collected infrared image of the cable trench according to the display information corresponding to the temperature in the infrared image, mark it, and use the area as the cable layout position.

[0206] The method for determining the edge of the cable based on the infrared image and the visible light image can be to fuse the infrared image and the visible light image of the same position, determine the distribution position of the cable through the temperature area in the infrared image, and determine the pixel points and pixel point features of the cable at the corresponding position in the visible light image, and determine the edge information of the cable based on the temperature difference between the cable and other positions in the infrared image and the pixel point features of the cable in the visible light image and the pixel point features of the cable at other positions.

[0207] The advantage of this arrangement of the present invention is that the heat generated by the cable during operation can be used to assist in determining the exact position of the cable in the cable trench through infrared images, so that the construction of the cable trench model is more in line with the actual situation.

[0208] Embodiment 8

[0209] Figure 8 Schematic diagram of the flow of the modeling method of the cable laying environment in the cable trench provided in the eighth embodiment of the present application. Figure 8As shown, the specific steps include:

[0210] S801, obtaining a visible light image in the cable trench through an RGB camera of an electronic device;

[0211] S802, acquiring a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device;

[0212] S803, identifying pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench;

[0213] S804, fusing the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench;

[0214] S805, obtaining the inspection operation track of the electronic equipment;

[0215] S806: construct a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0216] Optionally, identifying pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench includes:

[0217] Extracting semantic information from the visible light image, and distinguishing pixel points on the inner wall of the cable trench and pixel points on the cable according to the semantic information;

[0218] The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

[0219] In an embodiment of the present application, a visible light image in a cable trench is acquired through an RGB camera of an electronic device; a depth image in a cable trench is acquired through a depth camera of the electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device; pixel points in the visible light image and / or the depth image are identified to obtain pixel points on the inner wall of the cable trench; the visible light image and the depth image are fused to obtain a fusion result of the pixel points on the inner wall of the cable trench; an inspection operation trajectory of the electronic device is acquired; and a model of the cable trench is constructed according to the inspection operation trajectory and the fusion result of the pixel points on the inner wall of the cable trench to obtain a cable trench model.

[0220] Through the modeling device of the cable laying environment in the above-mentioned cable trench, a cable trench model can be created according to the visible light image and the depth image during the inspection process, so that the inspection personnel can fully grasp the actual situation in the cable trench, provide predictive information for the safe operation of the cable, and facilitate the timely discovery and handling of dangerous factors.

[0221] The modeling method for the cable laying environment in the cable trench provided in the embodiment of the present application corresponds to the modeling device for the cable laying environment in the cable trench provided in the above-mentioned embodiment, has the same functional modules and beneficial effects, and will not be repeated here to avoid repetition.

[0222] Embodiment 9

[0223] like Fig. 9 As shown, an embodiment of the present application also provides an electronic device 900, including a processor 901, a memory 902, and a program or instruction stored in the memory 902 and executable on the processor 901. When the program or instruction is executed by the processor 901, each process of the embodiment of the modeling device for the cable laying environment in the cable trench described above is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0224] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0225] Embodiment 10

[0226] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned embodiment of the modeling device for the cable laying environment in the cable trench are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0227] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0228] Embodiment 11

[0229] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the modeling device for the cable laying environment in the cable trench, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0230] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0231] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0232] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0233] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0234] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A modeling device for a cable laying environment in a cable trench, characterized in that: The device comprises: A visible light image acquisition module, used to acquire a visible light image in the cable trench through an RGB camera of an electronic device; A depth image acquisition module, used to acquire a depth image in the cable trench through a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device; A target recognition module, used to recognize pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench; A fusion module, used to fuse the visible light image and the depth image to obtain a fusion result of the pixel points on the inner wall of the cable trench; A patrol operation track acquisition module is used to acquire the patrol operation track of the electronic equipment; The model building module is used to build a model of the cable trench according to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench to obtain the cable trench model.

2. The device for modeling the cable laying environment in a cable trench according to claim 1, characterized in that: The target recognition module is specifically used for: Extracting semantic information from the visible light image, and distinguishing pixel points on the inner wall of the cable trench and pixel points on the cable according to the semantic information; The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

3. The device for modeling the cable laying environment in a cable trench according to claim 1, characterized in that: The target recognition module is specifically used for: Recognize the depth information in the depth image, and when it is recognized that the depth information has a sudden change and the sudden change meets a preset condition, recognize it as cable obstruction; In the case of cable shielding, the pixel points shielded by the cable are filtered out to obtain the pixel points on the inner wall of the cable trench.

4. The modeling device for the cable laying environment in a cable trench according to claim 2 or 3, characterized in that: The fusion module is further used for: Predicting depth data of the inner wall of the cable trench of the cable shielding portion according to the depth image; Predicting texture data of the inner wall of the cable trench of the cable shielding portion according to the visible light image; The inner wall of the filtered portion of the cable trench is filled based on the depth data and the texture data.

5. The device for modeling the cable laying environment in a cable trench according to claim 1, characterized in that: The model building module is also used to: When the inspection running track deviates, identifying whether the cable trench inner wall depth data changes with the running track deviation; If the depth data changes, obtain the visible light image at the position where the depth data changes, and extract the edge information therein; Abnormal depth data is identified and corrected based on the edge information.

6. The modeling device for the cable laying environment in the cable trench according to claim 1, characterized in that: The device further comprises: The cable location determination module is used to determine the distribution location of cables in the cable trench; Accordingly, the model building module is also used for: After the model building result is obtained, the distribution positions of the cables are marked on the cable trench model.

7. The device for modeling the cable laying environment in a cable trench according to claim 6, characterized in that: The cable position determination module is specifically used for: Acquire an infrared image in the cable trench, and determine the cable laying position according to the infrared image and the cable working temperature; The edge of the cable is determined according to the infrared image and the visible light image.

8. A method for modeling a cable laying environment in a cable trench, characterized in that: The method comprises: Obtain visible light images in the cable trench through the RGB camera of the electronic device; A depth image in the cable trench is obtained by a depth camera of an electronic device; wherein the depth camera and the RGB camera are arranged in parallel at a fixed position of the electronic device; Identify pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench; Fusing the visible light image and the depth image to obtain a fusion result of pixel points on the inner wall of the cable trench; Obtain the inspection and operation track of electronic equipment; According to the inspection running track and the fusion result of the pixel points on the inner wall of the cable trench, a model of the cable trench is constructed to obtain a cable trench model.

9. The method for modeling the cable laying environment in a cable trench according to claim 8, characterized in that: Identifying pixel points in the visible light image and / or the depth image to obtain pixel points on the inner wall of the cable trench includes: Extracting semantic information from the visible light image, and distinguishing pixel points on the inner wall of the cable trench and pixel points on the cable according to the semantic information; The cable pixels are filtered out to obtain the pixel points on the inner wall of the cable trench.

10. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the method for modeling the cable laying environment in the cable trench as described in any one of claims 8 to 9 are implemented.

Images