A pipeline corridor inspection method, device, electronic equipment and storage medium
Through virtual reality video streaming and three-dimensional model technology based on inspection robots, the problem of low efficiency and accuracy of pipe corridor inspection is solved, and intelligent and efficient pipe corridor inspection is realized, and coverage and accuracy are improved.
Patent Information
- Application Number
- CN202311390244.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-10-25
AI Technical Summary
The existing pipe corridor inspection methods have low efficiency and accuracy, high manual inspection intensity and are easily affected by skills and experience, online inspection is limited by the number of cameras and blind spots, and robot inspection accuracy is low.
Using a method based on inspection robot, by determining the inspection area in the inspection route of the pipeline corridor, obtaining virtual reality video streams, generating panoramic images, analyzing the inspection results to characterize the status of the pipeline corridor components, and combining three-dimensional models for labeling and overlaying display. , realize intelligent inspection.
It improves the efficiency and accuracy of pipeline inspection, solves battery life problems, saves costs, and has clear and intuitive real-life three-dimensional projection technology, and has comprehensive coverage of blind angles, reducing inspection time.
Smart Images

Figure CN117395374B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of pipeline corridor inspection, and in particular to a pipeline corridor inspection method, device, electronic equipment and storage medium. Background Art
[0002] In the field of tunnel inspection, existing tunnel inspection methods include manual inspection, online inspection and inspection robot. Manual inspection has the problems of high work intensity, low work efficiency and susceptibility to the influence of personnel skills and experience. Online inspection is limited by the number of cameras and monitoring blind spots in underground tunnels, and the efficiency of tunnel inspection is not high; robot inspection is mainly based on the analysis of collected images, and the accuracy of tunnel inspection is low. Therefore, the existing tunnel inspection methods have the problems of low efficiency and accuracy of tunnel inspection. Summary of the invention
[0003] Embodiments of the present application provide a pipeline corridor inspection method, device, electronic equipment and storage medium.
[0004] According to a first aspect of the present application, a pipeline corridor inspection method is provided, the method comprising: determining an inspection area in the pipeline corridor inspection route based on a pipeline corridor inspection route of an inspection robot; obtaining a virtual reality video stream corresponding to the inspection area based on the inspection robot; determining a first panoramic image corresponding to the inspection area based on the virtual reality video stream; and determining an inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image, the inspection result representing a status of components in an underground pipeline corridor corresponding to the pipeline corridor inspection route.
[0005] According to one embodiment of the present application, the tunnel inspection route based on the inspection robot determines the inspection area in the tunnel inspection route, including: the tunnel inspection route includes a charging route and a detection route; determining the position of the inspection robot; in response to the position of the inspection robot being in the charging route, charging the inspection robot; in response to the position of the inspection robot being in the detection route, determining the inspection area in the tunnel inspection route.
[0006] According to an embodiment of the present application, the virtual reality video stream corresponding to the inspection area is obtained based on the inspection robot, including: obtaining a first position in the pipeline corridor inspection route corresponding to the inspection area; in response to the inspection robot arriving at the first position, determining a sub-inspection area corresponding to the first position and a sampling angle range corresponding to the sub-inspection area; based on the sampling angle range, determining a first route for the inspection robot to inspect the sub-inspection area; based on the first route, obtaining a virtual reality video stream corresponding to the sub-inspection area.
[0007] According to one embodiment of the present application, determining a first route for the inspection robot to inspect the sub-inspection area based on the sampling angle range includes: determining each detection distance between each angle in the sampling angle range and the sub-inspection area; in response to the detection distance satisfying a preset detection condition, determining the corresponding detection distance as a first detection distance; based on the first detection distance and the corresponding angle, determining a first route for the inspection robot to inspect the sub-inspection area.
[0008] According to an implementation manner of the present application, determining the inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image includes: obtaining a reference panoramic image corresponding to the inspection area; performing a homography transformation on the first panoramic image based on the reference panoramic image to obtain a first difference area in the first panoramic image; determining a first attribute feature of a component included in the first difference area; obtaining a reference feature corresponding to the component included in the first difference area; and determining the status of the corresponding component based on the first attribute feature and the reference feature.
[0009] According to one embodiment of the present application, after determining the inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image, the method also includes: obtaining a three-dimensional model of the underground pipeline corridor corresponding to the pipeline corridor inspection route; annotating the three-dimensional model based on the status of components in the underground pipeline corridor included in the inspection result to obtain an annotated three-dimensional model; and superimposing and displaying the annotated three-dimensional model and the virtual reality video stream.
[0010] According to an embodiment of the present application, the superimposed display of the annotated three-dimensional model and the virtual reality video stream includes: importing the virtual reality video stream into a first layer; importing the annotated three-dimensional model into a second layer; calibrating the positions of the first layer and the second layer to obtain a superimposed reference point; based on the superimposed reference point, superimposing the first layer and the second layer to obtain a superimposed scene; and displaying the superimposed scene based on a preset first layer transparency and a preset second layer transparency.
[0011] According to a second aspect of the present application, a pipeline corridor inspection device is provided, which includes: a determination module for determining an inspection area in a pipeline corridor inspection route based on a pipeline corridor inspection route of an inspection robot; an acquisition module for acquiring a virtual reality video stream corresponding to the inspection area based on the inspection robot; a generation module for determining a first panoramic image corresponding to the inspection area based on the virtual reality video stream; and an inspection module for determining an inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image, wherein the inspection result represents a status of components in an underground pipeline corridor corresponding to the pipeline corridor inspection route.
[0012] According to a third aspect of the present application, an electronic device is provided, including:
[0013] at least one processor; and
[0014] a memory communicatively connected to the at least one processor; wherein,
[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.
[0016] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.
[0017] The method of the embodiment of the present application determines the inspection area in the pipeline corridor inspection route based on the pipeline corridor inspection route of the inspection robot; obtains the virtual reality video stream corresponding to the inspection area based on the inspection robot; determines the first panoramic view corresponding to the inspection area based on the virtual reality video stream; determines the inspection result corresponding to the pipeline corridor inspection route based on the first panoramic view, and the inspection result represents the state of the components in the underground pipeline corridor corresponding to the pipeline corridor inspection route. In this way, the pipeline corridor can be inspected intelligently, and the efficiency and accuracy of the pipeline corridor inspection are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:
[0019] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.
[0020] Figure 1 A schematic diagram of the processing flow of the pipeline corridor inspection method provided in an embodiment of the present application is shown;
[0021] Figure 2 A schematic diagram of the process of the pipeline corridor inspection method provided by the second embodiment of the present application is shown;
[0022] Figure 3 A schematic diagram of the process of the pipeline corridor inspection method provided in the third embodiment of the present application is shown;
[0023] Figure 4 A schematic diagram of a pipeline corridor inspection method provided in a fourth embodiment of the present application is shown;
[0024] Figure 5A schematic diagram of the process of the pipeline corridor inspection method provided in the fifth embodiment of the present application is shown;
[0025] Figure 6 A schematic diagram of a pipeline corridor inspection method provided in a sixth embodiment of the present application is shown;
[0026] Figure 7 An application scenario diagram of the pipeline corridor inspection method provided by an embodiment of the present application is shown;
[0027] Figure 8 An optional schematic diagram of a pipe gallery inspection device provided in an embodiment of the present application is shown;
[0028] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0029] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0030] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0031] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0033] The processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 1 , Figure 1 is a schematic diagram of the processing flow of the pipeline corridor inspection method provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S103 are shown for explanation.
[0034] Step S101, based on the pipeline corridor inspection route of the inspection robot, determine the inspection area in the pipeline corridor inspection route.
[0035] In some embodiments, the pipeline corridor inspection route can be a pre-set inspection track path of the inspection robot in the underground pipeline corridor. The pipeline corridor inspection route can be set in the underground space of the underground pipeline corridor. The pipeline corridor inspection route may include a charging route and a detection route. Among them, the charging route may include a power supply track, and the charging route can charge the inspection robot. A surrounding device is arranged on the periphery of the charging route to prevent the risk of electric shock to personnel. The detection route may not be connected to a power source. The inspection area in the underground pipeline corridor corresponding to the detection route can be pre-set. The inspection area may include various components in the underground pipeline corridor. Components may include walls, pipelines, brackets, etc. The embodiments of the present application do not limit the specific components in the underground pipeline corridor.
[0036] Step S102: Based on the inspection robot, a virtual reality video stream corresponding to the inspection area is obtained.
[0037] In some embodiments, the inspection robot may be provided with a set of binocular cameras and a monocular camera, and the binocular camera may be used to perform panoramic synthesis of left and right images to form a virtual reality (VR) video stream. The left camera in the binocular camera may be integrated with a radar for measuring distance, and the right camera may be integrated with an infrared sensing device for infrared imaging to calculate the calorific value of a heating object. The monocular camera may be used for video acquisition. An information transmission device may be provided at the contact point between the inspection robot and the track, and the information transmission device may be used to enable the inspection robot to interact with the track control system. The inspection robot may switch between different inspection methods according to the received track information. The inspection robot may be provided with various sensors, such as temperature and humidity sensors, oxygen sensors, smoke sensors, etc., which are used to receive the corresponding underground corridor information in the underground corridor. The inspection robot may be provided with a projection device, and the projection device is used to project the scene of the three-dimensional model into the real space, and the scene of the three-dimensional model may be marked.
[0038] Step S103: determining a first panoramic image corresponding to the inspection area based on the virtual reality video stream.
[0039] In some embodiments, the images captured by the left eye camera in the virtual reality video stream are analyzed and synthesized to obtain a left eye panoramic image, and the images captured by the right eye camera in the virtual reality video stream are analyzed and synthesized to obtain a right eye panoramic image. According to the mapping relationship between the left eye panoramic image and the right eye panoramic image, the left eye panoramic image and the right eye panoramic image are synthesized to obtain a first panoramic image. The first panoramic image can be a binocular panoramic image.
[0040] Step S104: Determine the inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image.
[0041] In some embodiments, the inspection results may include: structural safety inspection results, ventilation system inspection results, pipeline inspection results, other inspection results, etc. The embodiments of the present application do not limit the specific inspection results. The structural safety inspection results may include: whether cracks, deformations, corrosion, etc. occur in the components in the underground pipe gallery. If cracks, deformations, corrosion, etc. occur in the components, the structural safety inspection results also include the location information of the cracks, deformations, corrosion, etc. The inspection results of the ventilation system may include: whether the ventilation system of the underground pipe gallery is operating normally. If the ventilation system is abnormal, the inspection results of the ventilation system also include abnormal information of the abnormal ventilation system. The pipeline inspection results may include: the service life of various pipelines in the underground pipe gallery, the natural environment inspection information of the underground pipe gallery, such as groundwater level, soil pressure, etc. Other inspection results may include: the operating status of the equipment in the underground pipe gallery, the operating status of the power system, the temperature and humidity inside the underground pipe gallery, etc.
[0042] In some embodiments, the inspection results can also be compared with the sample pictures collected by the monocular camera in real time to obtain the inspection results. The sample pictures may include: gas pipeline sample pictures, water supply pipeline sample pictures, etc. The inspection results are marked in the 3D model with the corresponding points collected by the monocular camera, and the areas, attributes, and each sub-component are marked in the 3D model.
[0043] The method of the embodiment of the present application, through steps S101-S103, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0044] The specific processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 2 , Figure 2 A flow chart of the pipeline corridor inspection method according to the second embodiment of the present application is shown.
[0045] like Figure 2 As shown, the pipeline corridor inspection route based on the inspection robot in step S101 determines the inspection area in the pipeline corridor inspection route, including:
[0046] Step S201, determining the position of the inspection robot.
[0047] In this embodiment, the position of the inspection robot can be determined by a geographic information system (GIS). The inspection robot adopts an arm architecture and has a built-in Linux system, and the system runs on Linux.
[0048] Step S202a, in response to the inspection robot being located in a charging route, charging the inspection robot.
[0049] In this embodiment, the inspection robot can stop in the charging route to charge, and the inspection robot can also be charged in the charging route in a moving state.
[0050] Step S202b, in response to the inspection robot being located in the inspection route, determining the inspection area in the pipeline corridor inspection route.
[0051] In this embodiment, the inspection area in the underground pipe gallery corresponding to the detection route can be pre-set. The inspection area may include various components in the underground pipe gallery. The components may include walls, pipelines, brackets, etc. The embodiment of the present application does not limit the specific components in the underground pipe gallery. The location of the detection route may correspond to one or more inspection areas. The inspection area may include the area where the binocular camera of the inspection robot in the underground pipe gallery collects the virtual reality video stream.
[0052] As an example, a detection dead point position is pre-set in the detection route. The detection dead point position may include an object blocking the line between the inspection area corresponding to the detection dead point position and the binocular camera of the inspection robot. According to the detection dead point position in the detection route, the inspection area corresponding to the detection dead point position is determined.
[0053] The method of the embodiment of the present application, through steps S201-S202b, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0054] The specific processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 3 , Figure 3 A schematic flow chart of the pipeline corridor inspection method provided in the third embodiment of the present application is shown.
[0055] like Figure 3 As shown, the step S102 of obtaining a virtual reality video stream corresponding to the inspection area based on the inspection robot includes:
[0056] Step S301, obtain the first position in the pipeline corridor inspection route corresponding to the inspection area.
[0057] In this embodiment, the first position may be a detection dead point position in a preset pipeline corridor inspection route. The detection dead point position may correspond to one or more sub-inspection areas in the inspection area. The detection dead point position may include an object blocking the line between the inspection area corresponding to the detection dead point position and the binocular camera of the inspection robot.
[0058] Step S302, in response to the inspection robot arriving at a first position, determining a sub-inspection area corresponding to the first position and a sampling angle range corresponding to the sub-inspection area.
[0059] In this embodiment, the sub-inspection area may correspond to one or more preset sampling angle ranges of the binocular camera. The sampling angle range may be collected in advance according to the actual underground pipe gallery environment, and the angle range between the sub-inspection area and the binocular camera of the inspection robot is marked.
[0060] Step S303: determining a first route for the inspection robot to inspect the sub-inspection area based on the sampling angle range.
[0061] In this embodiment, the first route may be a moving route of a mechanical arm of the inspection robot. A binocular camera may be provided on the mechanical arm. The embodiment of the present application does not limit the specific moving route of the mechanical arm.
[0062] Step S304: Based on the first route, a virtual reality video stream corresponding to the sub-inspection area is obtained.
[0063] In this embodiment, the binocular camera can be arranged on the mechanical arm of the inspection robot. The mechanical arm can move along the first route so that the binocular camera can obtain the virtual reality video stream corresponding to the sub-inspection area.
[0064] The method of the embodiment of the present application, through steps S301-S304, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0065] The specific processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 4 , Figure 4 A flow chart of the pipeline corridor inspection method provided in the fourth embodiment of the present application is shown.
[0066] like Figure 4 As shown, in step S303, determining the first route for the inspection robot to inspect the sub-inspection area based on the sampling angle range includes:
[0067] Step S401, determining each detection distance between each angle in the sampling angle range and the sub-inspection area.
[0068] Step S402: in response to the detection distance satisfying a preset detection condition, determining the corresponding detection distance as a first detection distance.
[0069] In this embodiment, the preset detection condition may include: the difference between the detection distances is greater than a preset detection distance threshold.
[0070] Step S403: determining a first route for the inspection robot to inspect the sub-inspection area based on the first detection distance and the corresponding angle.
[0071] For steps S401-S403, the specific implementation may include: determining the starting angle Astart and the end angle Aend of the sampling angle range. The robot moves parallel to the Astart angle, and pauses to record the current robot position as T1 every time it moves 1 cm. The detection distance d1 between the binocular camera and the sub-inspection area at T1 is determined. The binocular camera rotates 90° to the left to obtain the detection distance corresponding to each angle, and then the binocular camera returns to its original position, and then rotates 90° to the right to obtain the detection distance corresponding to each angle, and then the binocular camera returns to its original position. These newly obtained detection distances are named d11, d12, d13, d14, d15, etc., and d1n is subtracted from d1 to obtain each difference. In response to the difference being greater than d1 / 10, the corresponding detection distance is determined as the first detection distance.
[0072] Determine the angle corresponding to the first detection distance and determine the deepest robot arm position corresponding to the first detection distance as T2, determine the detection distance d2 between the binocular camera and the sub-inspection area at T2, rotate the binocular camera 90° to the left to obtain the detection distance corresponding to each angle, then return the binocular camera, and then rotate it 90° to the right to obtain the detection distance corresponding to each angle, and then return the binocular camera to its position. Name these newly acquired detection distances d21, d22, d23, d24, d25, etc., and subtract d2n from d2 to obtain each difference. In response to the difference being greater than d2 / 10, determine the corresponding detection distance as the first detection distance. Determine the angle corresponding to the first detection distance and determine the robot arm position at the first detection distance as T3, and so on. Connect the robot arm positions T1, T2, T3, etc. to obtain the first route for the inspection robot to inspect the sub-inspection area.
[0073] The method of the embodiment of the present application, through steps S401-S403, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0074] The specific processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 5 , Figure 5 A flow chart of the pipeline corridor inspection method provided in the fifth embodiment of the present application is shown.
[0075] like Figure 5 As shown, the step S104 of determining the inspection result corresponding to the pipeline corridor inspection route based on the first panoramic view includes:
[0076] Step S501, obtaining a reference panoramic image corresponding to the inspection area.
[0077] In this embodiment, the reference panoramic image can be derived from a preset 3D model corresponding to the inspection area of the underground pipe gallery, and a reference angle of the 3D model is determined from the 3D rendering engine. The 3D model can be rotated 360 degrees.
[0078] In some embodiments, after step S501, the method further includes: determining a reference point 1 of the reference panorama, determining a reference point 2 of the first panorama, and performing position calibration and angle calibration on the reference panorama and the first panorama according to the reference point 1 and the reference point 2.
[0079] Step S502: Based on the reference panoramic image, perform homography transformation on the first panoramic image to obtain a first difference area in the first panoramic image.
[0080] In this embodiment, the first panoramic image may include multiple first feature points, and the reference panoramic image may include multiple reference feature points. In response to the difference between the feature value of the first feature point and the feature value of the reference feature point being greater than a preset difference threshold, the area composed of the corresponding first feature points is determined as the first difference area. The embodiment of the present application does not limit the specific difference threshold.
[0081] As an example, step S502 may be implemented by the following code:
[0082] Using [Python+OpenCv] to approximate image difference detection
[0083] #Detect key points
[0084] kp1,des1=sift.detectAndCompute(img1,None)
[0085] kp2,des2=sift.detectAndCompute(img2,None)
[0086] #Keypoint matching
[0087] FLANN_INDEX_KDTREE=0
[0088] index_params=dict(algorithm=FLANN_INDEX_KDTREE,trees=6)
[0089] search_params=dict(checks=10)
[0090] flann=cv2.FlannBasedMatcher(index_params,search_params)
[0091] matches=flann.knnMatch(des1,des2,k=2)
[0092] Step S503: determining a first attribute feature of the components included in the first difference region.
[0093] In this embodiment, the first attribute characteristics of the components included in the first difference area are determined according to the first feature point in the first difference area. The first attribute characteristics may include: structural safety attribute characteristics, ventilation system attribute characteristics, pipeline attribute characteristics, other attribute characteristics, etc. The embodiments of the present application do not limit the specific attribute characteristics. Structural safety attribute characteristics may include: component positions and component attributes in the underground pipe gallery. Ventilation system attribute characteristics may include: ventilation system operating status characteristics of the underground pipe gallery. Pipeline attribute characteristics may include: life characteristics and pipeline environment characteristics of various pipelines in the underground pipe gallery. Other attribute characteristics may include: equipment operating status characteristics in the underground pipe gallery, power system operating status characteristics, temperature characteristics and humidity characteristics inside the underground pipe gallery, etc.
[0094] Step S504: obtaining reference features corresponding to the components included in the first difference region.
[0095] In this embodiment, reference features are obtained from a preset reference feature database. The reference features may include: structural safety reference features, ventilation system reference features, pipeline reference features, other reference features, etc.
[0096] Step S505: determining the state of the corresponding component based on the first attribute feature and the reference feature.
[0097] In this example, each component corresponds to one or more feature determination methods, and according to the feature determination method corresponding to the component, it is determined whether the first attribute feature meets the reference feature, so as to determine the state of the component. This application does not limit the specific feature determination method.
[0098] As an example, the feature judgment method corresponding to the pipeline component is a water leakage detection method in the pipeline detection method, and the reference feature is that the water leakage aperture is greater than 1 cm and penetrates, or the fluid volume at the leaking location is greater than 1 cubic centimeter. In response to the first attribute feature meeting the reference feature, it is determined that the pipeline component is in a water leakage state, and the service life of the pipeline corresponding to the water leakage state is determined. In response to the first attribute feature not meeting the reference feature, it is determined that the pipeline component is in a normal state, and the service life of the pipeline corresponding to the normal state is determined.
[0099] The method of the embodiment of the present application, through steps S501-S505, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0100] The specific processing flow of the pipeline corridor inspection method provided in the embodiment of the present application is described. Figure 6 , Figure 6 A flow chart of the pipeline corridor inspection method provided in the sixth embodiment of the present application is shown.
[0101] like Figure 6 As shown, after step S104, the pipeline corridor inspection method further includes:
[0102] Step S601, obtaining a three-dimensional model of the underground tunnel corresponding to the tunnel inspection route.
[0103] Step S602: annotate the three-dimensional model based on the status of the components in the underground pipe gallery included in the inspection results to obtain the annotated three-dimensional model.
[0104] In this embodiment, the inspection results include the status of components in the underground pipe gallery, and the areas, attributes and components are marked in the three-dimensional model according to the corresponding positions.
[0105] Step S603: import the virtual reality video stream into the first layer.
[0106] Step S604: import the annotated three-dimensional model into the second layer.
[0107] Step S605: calibrate the positions of the first layer and the second layer to obtain a superposition reference point.
[0108] Step S606: superimpose the first layer and the second layer based on the superimposition reference point to obtain a superimposed scene.
[0109] Step S607: displaying the overlay scene based on the preset transparency of the first layer and the preset transparency of the second layer.
[0110] In this embodiment, the first layer of the superimposed scene is displayed according to the preset first layer transparency, and the second layer of the superimposed scene is displayed according to the preset second layer transparency.
[0111] For steps S601-S607, when implemented specifically, the steps may include: importing the virtual reality video stream acquired by the binocular camera into the first layer of the display engine by means of virtual construction. Importing the annotated three-dimensional model into the second layer of the display engine by means of virtual construction. Calibrate the positions of the first layer and the second layer to obtain a superimposed reference point, and based on the superimposed reference point, superimpose the first layer and the second layer to obtain a superimposed scene. Selectively control the transparency of a component or a layer to display the superimposed scene through the display engine.
[0112] The method of the embodiment of the present application, through steps S601-S607, can solve the problem of patrol robot endurance by laying tracks in the entire area and charging while patrolling, thereby saving the cost of corridor inspection. The difference algorithm detection technology of VR panorama and three-dimensional model-derived panorama can more accurately and comprehensively detect abnormalities in underground corridors, thereby improving the accuracy of corridor inspections. The real-scene three-dimensional projection technology is clearer and more intuitive than traditional paper-based inspections. At the same time, the blind spot optimization route can also solve the problem that the entire area of the underground corridor cannot be covered, thereby improving the accuracy of corridor inspections. The VR scene formed by binoculars and the three-dimensional model scene are superimposed for display, so the inspection content can be displayed more clearly and in a more contrasting manner, reducing the time occupied by corridor inspections, thereby improving the efficiency of corridor inspections.
[0113] Figure 7 An application scenario diagram of the pipeline corridor inspection method provided by an embodiment of the present application is shown;
[0114] refer to Figure 7 , an application scenario diagram of the pipeline corridor inspection method provided in an embodiment of the present application, which is applied to a pipeline corridor inspection robot device based on VR. The VR pipeline corridor inspection robot device includes: an inspection robot, an underground pipeline corridor inspection system and a background server. The underground pipeline corridor inspection system includes: a data collection module, a three-dimensional model processing module, a storage module, a calculation module, a communication module and a background monitoring module. Among them, the data collection module is used to collect real-time data from sensors and cameras.
[0115] The 3D model processing module is used to construct a 3D model according to the pipeline corridor standards, design specifications and real-life models; to perform benchmark calibration on the spatial positions of the 3D model and the binocular camera so that when the binocular camera moves, the 3D model can be moved and converted synchronously for display; and to superimpose the virtual reality video stream formed by the 3D model and the binocular camera.
[0116] The storage module is used to store the collected videos, pictures, and generated panoramic images according to time points and location points.
[0117] The calculation module is used to calculate the difference between the sensor's acquisition value and the preset threshold value to determine whether the component is faulty. The acquisition value will be stored in the database in real time for horizontal and vertical comparison. Horizontal comparison refers to the comparison of sensor values in different time periods, and vertical comparison refers to the comparison of values between different pipe galleries. The comparison operation of video acquisition can be divided into four types: 1. Comparison of video acquisition images with sample images of the same attributes. 2. Comparison of the VR video stream formed by the binocular camera with the screenshot of the same angle position of the three-dimensional model. 3. Projection comparison of the projection device projected into the real scene. 4. Change detection and analysis of the panoramic image corresponding to the VR video stream and the panoramic image corresponding to the three-dimensional model.
[0118] The communication module is used to communicate with the backend server and transmit the detection results, video comparison images, videos and other data to the backend server in real time.
[0119] The background monitoring module is used to receive the inspection information, real-time video information, and infrared imaging information transmitted by the inspection robot, and present a visual overlay scene on the user side.
[0120] Understandably, Figure 7 The application scenarios of the pipeline inspection method are only some exemplary implementations in the embodiments of the present application. The application scenarios of the pipeline inspection method in the embodiments of the present application include but are not limited to Figure 7 The application scenario of the pipeline corridor inspection method shown.
[0121] The following is a description of an exemplary structure of the pipeline corridor inspection device 90 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 8 As shown, the software modules in the tunnel inspection device 90 may include: a determination module 901, which is used to determine the inspection area in the tunnel inspection route based on the tunnel inspection route of the inspection robot; an acquisition module 902, which is used to acquire the virtual reality video stream corresponding to the inspection area based on the inspection robot; a generation module 903, which is used to determine the first panoramic image corresponding to the inspection area based on the virtual reality video stream; and an inspection module 904, which is used to determine the inspection result corresponding to the tunnel inspection route based on the first panoramic image, and the inspection result represents the status of the components in the underground tunnel corresponding to the tunnel inspection route.
[0122] It should be noted that the description of the device in the embodiment of the present application is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 7 The present invention can be understood by referring to the description of any one of the accompanying drawings.
[0123] According to an embodiment of the present application, the present application also provides an electronic device and a non-transitory computer-readable storage medium.
[0124] Fig. 9 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0125] like Fig. 9 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0126] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0127] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the pipe gallery inspection method. For example, in some embodiments, the pipe gallery inspection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the pipe gallery inspection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the pipe gallery inspection method in any other appropriate manner (e.g., by means of firmware).
[0128] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0129] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0130] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0132] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0133] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0135] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A pipeline corridor inspection method, characterized in that: The method comprises: Based on the pipeline corridor inspection route of the inspection robot, determine the inspection area in the pipeline corridor inspection route; Based on the inspection robot, a virtual reality video stream corresponding to the inspection area is obtained; Determining a first panoramic image corresponding to the inspection area based on the virtual reality video stream; Based on the first panoramic image, determining an inspection result corresponding to the pipeline corridor inspection route, wherein the inspection result represents a state of components in the underground pipeline corridor corresponding to the pipeline corridor inspection route; The method of determining the inspection result corresponding to the inspection route of the pipe gallery based on the first panoramic image includes: obtaining a reference panoramic image corresponding to the inspection area; performing a homography transformation on the first panoramic image based on the reference panoramic image to obtain a first difference area in the first panoramic image; determining a first attribute feature of a component included in the first difference area; obtaining a reference feature corresponding to the component included in the first difference area; determining the state of the corresponding component based on the first attribute feature and the reference feature; the first attribute feature includes: a structural safety attribute feature, a ventilation system attribute feature and a pipeline attribute feature; the reference feature includes: a structural safety reference feature, a ventilation system reference feature and a pipeline reference feature; Obtaining a three-dimensional model of the underground pipeline corridor corresponding to the pipeline corridor inspection route; Based on the status of the components in the underground pipe gallery included in the inspection results, the three-dimensional model is annotated to obtain a annotated three-dimensional model; The annotated three-dimensional model and the virtual reality video stream are displayed in a superimposed manner.
2. According to the method of claim 1, the pipeline corridor inspection route based on the inspection robot, determining the inspection area in the pipeline corridor inspection route, comprises: The pipeline corridor inspection route includes a charging route and a detection route; Determining the position of the inspection robot; In response to the inspection robot being located in the charging route, charging the inspection robot; In response to the inspection robot being located in the inspection route, an inspection area in the pipeline corridor inspection route is determined.
3. According to the method of claim 1, obtaining the virtual reality video stream corresponding to the inspection area based on the inspection robot comprises: Obtaining a first position in the pipeline corridor inspection route corresponding to the inspection area; In response to the inspection robot arriving at the first position, determining a sub-inspection area corresponding to the first position and a sampling angle range corresponding to the sub-inspection area; Based on the sampling angle range, determining a first route for the inspection robot to inspect the sub-inspection area; Based on the first route, a virtual reality video stream corresponding to the sub-inspection area is obtained.
4. The method according to claim 3, wherein determining a first route for the inspection robot to inspect the sub-inspection area based on the sampling angle range comprises: Determine each detection distance between each angle in the sampling angle range and the sub-inspection area; In response to the detection distance satisfying a preset detection condition, determining the corresponding detection distance as a first detection distance; Based on the first detection distance and the corresponding angle, a first route for the inspection robot to inspect the sub-inspection area is determined.
5. According to the method of claim 1, the superimposed display of the annotated 3D model and the virtual reality video stream comprises: Importing the virtual reality video stream into the first layer; Importing the annotated three-dimensional model into the second layer; Performing position calibration on the first layer and the second layer to obtain a superposition reference point; Based on the superposition reference point, superimpose the first layer and the second layer to obtain a superimposed scene; The overlay scene is displayed based on a preset first layer transparency and a preset second layer transparency.
6. A pipe gallery inspection device, characterized in that: The device comprises: A determination module, used to determine the inspection area in the pipeline corridor inspection route based on the pipeline corridor inspection route of the inspection robot; An acquisition module, used for acquiring a virtual reality video stream corresponding to the inspection area based on the inspection robot; A generating module, configured to determine a first panoramic image corresponding to the inspection area based on the virtual reality video stream; An inspection module, configured to determine an inspection result corresponding to the pipeline corridor inspection route based on the first panoramic image, wherein the inspection result represents a state of components in the underground pipeline corridor corresponding to the pipeline corridor inspection route; The method of determining the inspection result corresponding to the inspection route of the pipe gallery based on the first panoramic image includes: obtaining a reference panoramic image corresponding to the inspection area; performing a homography transformation on the first panoramic image based on the reference panoramic image to obtain a first difference area in the first panoramic image; determining a first attribute feature of a component included in the first difference area; obtaining a reference feature corresponding to the component included in the first difference area; determining the state of the corresponding component based on the first attribute feature and the reference feature; the first attribute feature includes: a structural safety attribute feature, a ventilation system attribute feature and a pipeline attribute feature; the reference feature includes: a structural safety reference feature, a ventilation system reference feature and a pipeline reference feature; Obtaining a three-dimensional model of the underground pipeline corridor corresponding to the pipeline corridor inspection route; Based on the status of the components in the underground pipe gallery included in the inspection results, the three-dimensional model is annotated to obtain a annotated three-dimensional model; The annotated three-dimensional model and the virtual reality video stream are displayed in a superimposed manner.
7. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.
