Control method of mobile equipment, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a control method of mobile equipment, electronic equipment and a readable storage medium, relates to the field of unmanned aerial vehicle inspection, and can improve the safety of autonomous flight of an unmanned aerial vehicle along a power transmission line. Comprising the steps of obtaining two-dimensional perception information and three-dimensional perception information of a pipeline network under the same space reference through mobile equipment, and determining a current reliability quantized value of the perception capability of the mobile equipment based on the two-dimensional perception information, the three-dimensional perception information and uniqueness judgment results of all three-dimensional feature points, different reliability quantized values correspond to different flight distance constraint conditions, and the flight distance constraint conditions are used for constraining the transverse and longitudinal flight distances of the mobile device to the detection pipeline, and determining the available pipeline length based on the target flight distance constraint condition corresponding to the current reliability quantized value and the available pipeline length corresponding to the pipeline network. And determining a target control instruction of the mobile equipment.

Description

Technical Field

[0001] This application relates to the field of drone inspection, and more particularly to a control method for a mobile device, an electronic device, and a readable storage medium. Background Technology

[0002] In existing technologies, drones achieve autonomous flight along power transmission lines by continuously sensing the results of power transmission lines in complex outdoor environments, thereby enabling the inspection of power transmission lines.

[0003] However, the existing continuous sensing results of drones are affected by complex and harsh environments and complex power transmission lines, resulting in low reliability and relatively low safety for drones flying autonomously along power transmission lines. Summary of the Invention

[0004] This application provides a control method for a mobile device, an electronic device, and a readable storage medium that can improve the safety of unmanned aerial vehicles (UAVs) flying autonomously along power transmission lines.

[0005] To achieve the above objectives, this application adopts the following technical solution: In a first aspect, this application provides a method for controlling a mobile device, the method comprising: The mobile device acquires two-dimensional and three-dimensional perception information of a pipeline network under the same spatial reference. The pipeline network comprises multiple detection pipelines. The two-dimensional perception information includes image sequences of these pipelines, and the three-dimensional perception information includes the uniqueness determination results of the three-dimensional feature points corresponding to each detection pipeline. Based on the two-dimensional and three-dimensional perception information and the uniqueness determination results of each three-dimensional feature point, the current reliability quantification value of the mobile device's perception capability is determined. Different reliability quantification values ​​correspond to different flight distance constraints, which constrain the lateral and longitudinal flight distances of the mobile device relative to the detection pipelines. Based on the target flight distance constraints corresponding to the current reliability quantification value and the available pipeline lengths corresponding to the pipeline network, the target control command for the mobile device is determined.

[0006] Based on the first aspect, two-dimensional perception information can characterize two-dimensional image information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network; three-dimensional perception information can characterize three-dimensional information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network. The uniqueness determination result of the detected pipes included in the three-dimensional perception information can characterize the probability that each three-dimensional feature point in the three-dimensional perception information can represent the information of the same target detected pipe. Combining two-dimensional and three-dimensional perception information can obtain a more accurate actual perception situation of the detected pipes (i.e., a quantified value of the reliability of the perception capability), and the flight distance constraint of the mobile device determined based on the more accurate actual perception situation is also more accurate. Furthermore, the available pipe length refers to the length of the effectively detected pipes, which are unobstructed. The mobile device performs detection within the available pipe length; for unavailable pipes, i.e., other pipes affected by adverse environments or obstructed by other pipes, the mobile device does not perform detection. When the mobile device moves to other pipes, the flight distance between the mobile device and other pipes is relatively far, thus avoiding the impact of complex and adverse environments and complex pipe networks on the flight of the mobile device and the collection of perception data. Furthermore, the commands generated based on relatively accurate flight distance constraints and available pipe length are also relatively accurate. Thus, the mobile device can achieve relatively accurate and reliable control during its flight along the detection pipe, thereby improving the safety and continuity of the mobile device's autonomous flight.

[0007] In one feasible implementation, the determination of the current reliability quantification value of the mobile device's sensing capability based on the uniqueness determination results of two-dimensional sensing information, three-dimensional sensing information, and each three-dimensional feature point includes: Based on two-dimensional and three-dimensional perception information, the projection overlap ratio is obtained. The projection overlap ratio, two-dimensional perception information, three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point are input into the perception reliability quantification model, and the current reliability quantification value of the mobile device's perception capability is output.

[0008] In another feasible implementation, the aforementioned three-dimensional sensing information includes the pipe centerline of the detected pipe and the three-dimensional sensing range, while the two-dimensional sensing information includes the two-dimensional sensing range.

[0009] The above-mentioned acquisition of the projection overlap ratio based on two-dimensional and three-dimensional perception information includes: The three-dimensional sensing information is fitted into a three-dimensional guide. Under the same spatial reference, based on the pipe centerline and the three-dimensional sensing range corresponding to the three-dimensional sensing information, the three-dimensional guide is projected onto the two-dimensional sensing range corresponding to the two-dimensional sensing information to obtain two-dimensional projection information. Based on the two-dimensional projection information and the two-dimensional sensing information, the coincident guide is determined and the projection coincidence ratio is obtained.

[0010] In another feasible implementation, the aforementioned perceptual reliability quantification model includes a two-dimensional feature extraction network, a three-dimensional feature extraction network, and a fusion network.

[0011] The above-mentioned projection overlap ratio, two-dimensional perception information, three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point are input into the perception reliability quantification model, and the output is the current reliability quantification value of the mobile device's perception capability, including: Two-dimensional (2D) sensing information is input into a 2D feature extraction network to obtain the corresponding 2D feature embedding vector. Similarly, three-dimensional (3D) sensing information is input into a 3D feature extraction network to obtain the corresponding 3D feature embedding vector. The 2D feature embedding vector, the 3D feature embedding vector, the projection overlap ratio, and the uniqueness determination result of each 3D feature point are input into a fusion network to obtain a fused feature vector. Based on the fused feature vector, the current reliability quantification value of the mobile device's sensing capability is determined.

[0012] In another feasible implementation, the two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination result of each three-dimensional feature point are input into the fusion network to obtain the fused feature vector, including: Calculate the difference vector and product vector between the two-dimensional feature embedding vector and the three-dimensional feature embedding vector. Then, concatenate and fuse the difference vector, product vector, two-dimensional feature embedding vector, three-dimensional feature embedding vector, projection overlap ratio, and uniqueness determination results of each three-dimensional feature point to obtain the fused feature vector.

[0013] In another feasible implementation, the aforementioned perceived reliability quantification model also includes a decision network.

[0014] The above-mentioned determination of the current reliability quantification value of the mobile device's sensing capability based on fused feature vectors includes: The fused feature vector is input into the decision network to obtain the current reliability quantization value and the anomaly type of the current reliability quantization value corresponding to the perception capability of the mobile device.

[0015] In another feasible implementation, the lower the current reliability quantification of the sensing capability of the aforementioned mobile device, the greater the flight distance indicated by the flight distance constraint.

[0016] In another feasible implementation, the above method also includes: If the available pipeline length is greater than or equal to the preset length threshold, and the current reliability quantification value is within the first preset range, increase the flight distance indicated in the target flight distance constraint condition corresponding to the current reliability quantification value to obtain the updated target flight distance constraint condition. If the available pipeline length is less than a preset length threshold, and the current reliability quantification value is within a first preset range, the target flight distance constraint corresponding to the current reliability quantification value remains unchanged.

[0017] In another feasible implementation, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value is increased to obtain the updated target flight distance constraint, including: Based on the anomaly type corresponding to the current reliability quantification value, determine the adjustment direction and adjustment range; Based on the adjustment direction and adjustment range, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value is increased to obtain the updated target flight distance constraint.

[0018] In another feasible implementation, the above-mentioned anomaly types include two-dimensional perception information anomalies, three-dimensional perception information anomalies, and overlap anomalies, and the adjustment directions include horizontal and vertical directions.

[0019] The above determines the adjustment direction and magnitude based on the anomaly type corresponding to the current reliability quantification value, including: If the anomaly type corresponding to the current reliability quantification value is a two-dimensional perception information anomaly, the adjustment direction corresponding to the two-dimensional perception information anomaly is determined to be vertical, and the adjustment range corresponding to the vertical direction is a preset first vertical range. If the anomaly type corresponding to the current reliability quantification value is a three-dimensional perception information anomaly, the adjustment direction corresponding to the three-dimensional perception information anomaly is determined to be horizontal, and the adjustment range corresponding to the horizontal direction is a preset second horizontal range. If the anomaly type corresponding to the current reliability quantification value is an overlapping anomaly, the adjustment direction corresponding to the overlapping anomaly is determined to be both horizontal and vertical, the adjustment range corresponding to the horizontal direction is a preset third horizontal range, and the adjustment range corresponding to the vertical direction is a preset third vertical range.

[0020] In another feasible implementation, the above method also includes: If the projection overlap ratio is greater than or equal to the preset overlap threshold, the preset flight distance constraint is used as the target flight distance constraint. If the projection overlap ratio is less than the preset overlap threshold, the target flight distance constraint corresponding to the current reliability quantification value remains unchanged.

[0021] In another feasible implementation, the acquisition of two-dimensional and three-dimensional sensing information of the pipeline network under the same spatial reference via mobile devices includes: Initial 2D sensing information is acquired through a 2D sensing device on a mobile device, including an image acquisition device, and the initial 2D sensing information includes an image sequence of a pipe network. Initial 3D sensing information is acquired through a 3D sensing device on the mobile device, including a LiDAR device, and the initial 3D sensing information includes point cloud data of the pipe network. The initial 2D and 3D sensing information are mapped to the same spatial reference to determine the 2D sensing range corresponding to the initial 2D sensing information of the pipe centerline and the 3D sensing range corresponding to the initial 3D sensing information. Based on the pipe centerline and the 2D sensing range, the detected pipes in the image sequence are segmented into pipe segments to obtain a first pipe segment set, which includes multiple image sequence segments. Based on the pipe centerline and the 3D sensing range, the detected pipes in the point cloud data are segmented into pipe segments to obtain a second pipe segment set, which includes multiple point cloud data segments. Based on the first pipe segment set, 2D sensing information is acquired, and based on the second pipe segment set, 3D sensing information is acquired.

[0022] In another feasible implementation, the two-dimensional sensing information includes directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers.

[0023] The above-mentioned acquisition of two-dimensional sensing information based on the first pipeline segment set includes: Based on the direction of the pipeline centerline, the directional consistency quantization value corresponding to each image sequence segment in the first pipeline segment set is obtained. Based on the wire pixels of the image sequence segments along the pipeline centerline direction, the longest continuous length and the number of breaks corresponding to each image sequence segment in the first pipeline segment set are obtained. Based on preset inflection point determination conditions, the inflection point markers corresponding to each image sequence segment in the first pipeline segment set are determined. Based on the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers of each image sequence segment in the first pipeline segment set, the two-dimensional perception information corresponding to the first pipeline segment set is determined.

[0024] In another feasible implementation, the above-mentioned determination of the inflection point markers corresponding to each image sequence segment of the first pipeline segment set based on preset inflection point determination conditions includes: If the direction change rate of an image sequence segment is greater than or equal to a preset direction change rate threshold, the occlusion segment of each image sequence segment in the first pipeline segment set is determined. Based on the occlusion segment, the occlusion arc length of each image sequence segment in the first pipeline segment set is obtained. Based on the occlusion arc length, the inflection point marker corresponding to each image sequence segment in the first pipeline segment set is determined.

[0025] In another feasible implementation, the three-dimensional sensing information includes a first distribution uniformity quantization value, an echo stability quantization value, and a first fine-tuning stability quantization value.

[0026] The above-mentioned acquisition of three-dimensional perception information based on the second pipeline segment set includes: Based on the direction of the pipeline centerline, the number of 3D feature points in each point cloud data segment of the second pipeline segment set within a preset length is determined, obtaining the first distribution uniformity quantization value for each point cloud data segment in the second pipeline segment set. Based on the echo intensity of each point cloud data segment in the second pipeline segment set, the echo stability quantization value for each point cloud data segment in the second pipeline segment set is determined. Based on a preset fine-tuning amplitude and the 3D sensing range of the point cloud data segments, the first fine-tuning stability quantization value for each point cloud data segment in the second pipeline segment set is obtained. Based on the first distribution uniformity quantization value, the echo stability quantization value, and the first fine-tuning stability quantization value of each point cloud data segment in the second pipeline segment set, the 3D sensing information corresponding to the second pipeline segment set is determined.

[0027] In another feasible implementation, the above method also includes: Based on a preset offset pipeline centerline parallel to the pipeline centerline, the second distribution uniformity quantization value and the second fine-tuning stability quantization value of each point cloud data segment in the offset second pipeline segment set are obtained. Based on the first distribution uniformity quantization value, the first fine-tuning stability quantization value, the second distribution uniformity quantization value, and the second fine-tuning stability quantization value, the unique quantization value of each point cloud data segment in the second pipeline segment set is obtained through a preset unique quantization formula.

[0028] In another feasible implementation, the above method also includes: If the uniqueness quantization value is greater than or equal to the uniqueness quantization threshold, the uniqueness determination result for each point cloud data segment in the second pipeline segment set is determined to be "yes," indicating that the reliability of each point cloud data segment in the second pipeline segment set is high. If the uniqueness quantization value is less than the uniqueness quantization threshold, the uniqueness determination result for each point cloud data segment in the second pipeline segment set is determined to be "no," indicating that the reliability of each point cloud data segment in the second pipeline segment set is low.

[0029] In another feasible implementation, the above method also includes: If the current reliability quantification value is within the second preset range, the projection overlap ratio is greater than or equal to the preset overlap threshold, and the uniqueness determination result is yes, then determine the available pipeline length corresponding to the pipeline network.

[0030] Secondly, this application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the above-mentioned control method for the mobile device.

[0031] Thirdly, this application provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the above-mentioned mobile device control method.

[0032] Fourthly, this application provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the aforementioned method for controlling a mobile device.

[0033] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This application scenario diagram illustrates a control method for a mobile device provided in an embodiment of this application. Figure 2 An application scenario diagram illustrating another mobile device control method provided in this application embodiment; Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application; Figure 4 A flowchart illustrating a control method for a mobile device provided in an embodiment of this application; Figure 5 A schematic diagram of a process for obtaining a pipeline segment set is provided in an embodiment of this application; Figure 6 This application provides a schematic diagram of a process for obtaining two-dimensional sensing information corresponding to a first pipeline segment set. Figure 7 This application provides a schematic diagram of a process for obtaining three-dimensional perception information corresponding to a second pipeline segment set. Figure 8 A schematic diagram of a process for obtaining reliability quantification values ​​based on a perceived reliability quantification model is provided for an embodiment of this application; Figure 9 A network architecture diagram of a perceived reliability quantification model provided in this application embodiment; Figure 10 This is a partial inspection schematic diagram of an inspection pipeline provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a control device for a mobile device provided in an embodiment of this application. Detailed Implementation

[0036] The following describes in detail, with reference to the accompanying drawings, a method for controlling a mobile device according to an embodiment of this application.

[0037] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0038] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0039] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0040] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0041] The following explanations of some terms used in the embodiments of this disclosure are provided to facilitate understanding by those skilled in the art: (1) Transmission line inspection: to conduct status detection, defect investigation and hidden danger management of transmission lines to avoid power outages caused by line faults and potential risks.

[0042] (2) Camera image sequence: In the field of power transmission line inspection, camera image sequence refers to a set of images of the target power transmission line that are continuously collected by the inspection equipment equipped with a camera at fixed time intervals or spatial intervals and have temporal correlation.

[0043] (3) LiDAR point cloud: In the field of power transmission line inspection, LiDAR point cloud refers to the set of three-dimensional coordinate points formed by the inspection equipment carrying LiDAR, emitting laser beams towards the target power transmission line, and calculating the distance, angle and other information between the target power transmission line and the sensor by receiving the reflected signals.

[0044] In existing technologies, drones achieve autonomous flight along power transmission lines by continuously sensing the results of power transmission lines in complex outdoor environments, thereby enabling the inspection of power transmission lines.

[0045] However, the existing continuous sensing results of drones are affected by complex and harsh environments and complex power transmission lines, resulting in low reliability and relatively low safety for drones flying autonomously along power transmission lines.

[0046] To address the aforementioned technical issues, the mobile device control method provided in this application acquires two-dimensional and three-dimensional perception information of the pipeline network under the same spatial reference through the mobile device. Based on the two-dimensional and three-dimensional perception information and the uniqueness determination results of each three-dimensional feature point, the current reliability quantification value of the mobile device's perception capability is determined. Different reliability quantification values ​​correspond to different flight distance constraints. The flight distance constraints are used to constrain the lateral and / or longitudinal flight distance of the mobile device for detecting the pipeline. Based on the target flight distance constraints corresponding to the current reliability quantification value and the available pipeline length corresponding to the pipeline network, the target control command of the mobile device is determined.

[0047] In this embodiment, two-dimensional perception information can characterize two-dimensional image information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network; three-dimensional perception information can characterize three-dimensional information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network; wherein, the uniqueness determination result of the detected pipes included in the three-dimensional perception information can characterize the probability that each three-dimensional feature point in the three-dimensional perception information can represent the information of the same target detected pipe. Combining two-dimensional and three-dimensional perception information can obtain a more accurate actual perception situation of the detected pipes (i.e., the reliability quantification value of the perception capability), and the flight distance constraint of the mobile device determined based on the more accurate actual perception situation is also more accurate. Further, the available pipe length refers to the length of the effective detected pipes, and the effective detected pipes refer to the detected pipes without obstruction. The mobile device performs detection within the available pipe length; for unavailable pipes, that is, for other pipes affected by harsh environments or obstruction, the mobile device does not perform detection. When the mobile device moves to other pipes, the flight distance between the mobile device and other pipes is relatively far, thereby avoiding the impact of complex and harsh environments and complex pipe networks on the flight of the mobile device and the collection of perception data. Furthermore, the commands generated based on relatively accurate flight distance constraints and available pipe length are also relatively accurate. Thus, the mobile device can achieve relatively accurate and reliable control during its flight along the detection pipe, thereby improving the safety and continuity of the mobile device's autonomous flight.

[0048] In one embodiment of this application, the executing entity of the mobile device control method provided in this application can be an electronic device. For example, the electronic device can be a computing device or a server. The server can be a single physical server; or it can be two or more physical servers sharing different responsibilities, with multiple servers cooperating to achieve the corresponding functions. According to the server type, for example, the server can be a blade server, a high-density server, a rack server, or a tower server, etc.

[0049] In another scenario, the device executing the mobile device control method provided in this application can also be a mobile device, also known as an inspection device. For example, the inspection device could be a drone, an inspection robot, a helicopter, an inspection vehicle, etc.

[0050] Taking a mobile device as the executing entity as an example, Figure 1 This is an application scenario diagram of a mobile device control method provided in an embodiment of this application.

[0051] like Figure 1 As shown, Figure 1It includes a pipeline network 101, a target detection pipeline 102, a mobile device 103, and a power transmission tower 104. Among them, the mobile device 103 is equipped with a two-dimensional sensing device 1031 and a three-dimensional sensing device 1032.

[0052] In this embodiment, the pipeline network 101 refers to a network composed of multiple detection pipelines, and the starting position, route direction, ending position, and surrounding environment of different detection pipelines are not completely the same. For example, the pipelines can be power transmission lines, oil and gas pipelines, etc.

[0053] Among them, the target detection pipeline 102 refers to the target object to be sensed by the two-dimensional sensing device 1031 and the three-dimensional sensing device 1032 in the mobile device 103.

[0054] The two-dimensional sensing device 1031 can be any device capable of outputting two-dimensional data such as the appearance, temperature distribution, and planar position information of the target detection pipeline. For example, the two-dimensional sensing device can be an image acquisition device, a visible light camera, an infrared thermal imager, an ultraviolet imager, etc.

[0055] The 3D sensing device 1032 can be any device capable of outputting 3D data of a target detection pipeline. For example, the 3D sensing device can be a LiDAR device, a structured light 3D scanner, a binocular stereo camera, etc.

[0056] The mobile device 103 acquires initial two-dimensional perception information of the target detection pipeline 102 through the two-dimensional perception device 1031 mounted on the mobile device 103, and acquires initial three-dimensional perception information of the target detection pipeline 102 through the three-dimensional perception device 1032 mounted on the mobile device 103. Based on the initial two-dimensional perception information and the initial three-dimensional perception information, the mobile device determines the target control command of the mobile device, so that the mobile device flies autonomously along the target detection pipeline based on the target control command.

[0057] Taking electronic devices as the executing entity as an example, Figure 2 This is an application scenario diagram of another mobile device control method provided in an embodiment of this application.

[0058] like Figure 2 The figure includes an electronic device 200, a pipeline network 101, a target detection pipeline 102, a mobile device 103, and a power transmission tower 104. The mobile device 103 is equipped with a two-dimensional sensing device 1031 and a three-dimensional sensing device 1032.

[0059] The electronic device determines the target control command of the mobile device based on the initial two-dimensional perception information of the target detection pipeline 102 collected by the two-dimensional perception device 1031 on the mobile device 103 and the initial three-dimensional perception information of the target detection pipeline 102 collected by the three-dimensional perception device 1032 on the mobile device 103. The electronic device sends the target control command to the mobile device, so that the mobile device flies autonomously along the target detection pipeline based on the target control command.

[0060] Taking electronic devices as an example, Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. See also... Figure 3 , Figure 3 The electronic device 300 shown may include a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, the memory 302, and the communication interface 303 can be connected via the bus 304.

[0061] The processor 301 is the control center of the electronic device. It can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0062] As an example, processor 301 may include one or more CPUs, for example Figure 3 CPU 0 and CPU 1 are shown in the diagram.

[0063] The memory 302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0064] In one possible implementation, the memory 302 may exist independently of the processor 301. The memory 302 can be connected to the processor 301 via a bus 304 and is used to store data, instructions, or program code. When the processor 301 calls and executes the instructions or program code stored in the memory 302, it can implement the mobile device control method provided in the embodiments of this application.

[0065] In another possible implementation, the memory 302 can also be integrated with the processor 301.

[0066] The communication interface 303 is used for the first communication device to connect with other devices via a communication network, which may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 303 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0067] Bus 304 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0068] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on the electronic device, except... Figure 3 In addition to the components shown, the electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0069] The following examples illustrate the concept of a mobile device as the execution subject.

[0070] Figure 4 This is a flowchart illustrating a control method for a mobile device provided in an embodiment of this application, as shown below. Figure 4 As shown, the method includes: S401. Obtain two-dimensional and three-dimensional sensing information of the pipeline network under the same spatial reference through mobile devices.

[0071] The pipeline network comprises multiple detection pipelines, each with different starting points, routes, ending points, and surrounding environments. In one scenario, multiple detection pipelines share the same starting and ending points, but their routes and surrounding environments differ. In another scenario, multiple detection pipelines share the same starting point, route, and ending point; that is, multiple detection pipelines operate in parallel, but their surrounding environments are not entirely identical.

[0072] Two-dimensional and three-dimensional perceptual information under the same spatial reference represent that both two-dimensional and three-dimensional perceptual information are in the same space.

[0073] In one scenario, two-dimensional and three-dimensional sensing information under the same spatial reference can both be located in the spatial region corresponding to the three-dimensional sensing information.

[0074] In another scenario, two-dimensional and three-dimensional sensing information under the same spatial reference can both be located in the spatial region corresponding to the two-dimensional sensing information.

[0075] In another scenario, two-dimensional and three-dimensional perceptual information under the same spatial reference can both be located in a pre-defined spatial region.

[0076] Two-dimensional sensing information is used to characterize the condition of detected pipelines. It includes image sequences of multiple detected pipelines within a pipeline network, as well as quantified values ​​of directional consistency, longest continuous length, number of breaks, occlusion sections, and inflection point markers.

[0077] Three-dimensional sensing information is used to characterize the three-dimensional information of the detected pipeline. This information includes the uniqueness determination result of the three-dimensional feature points corresponding to the detected pipeline. The uniqueness determination result is used to characterize the probability that each three-dimensional feature point in the second pipeline set can represent the information of the same target detected pipeline. The three-dimensional sensing information also includes quantized values ​​for distribution uniformity, echo stability, and fine-tuning stability.

[0078] S402. Based on the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point, determine the current reliability quantification value of the mobile device's perception capability.

[0079] The reliability quantification value characterizes the degree to which the detection pipeline can be accurately perceived. The reliability quantification value can be a natural number greater than 0. In one implementation, the reliability quantification value can be normalized; for example, the range of the reliability quantification value can be [0, 1]. The closer the reliability quantification value is to 1, the more reliable the mobile device's perception of the detection pipeline, and the more accurate the perception result. Correspondingly, the easier it is for the mobile device to perceive the detection pipeline. Conversely, the closer the reliability quantification value is to 0, the weaker the mobile device's perception of the detection pipeline, and the less accurate the perception result. Correspondingly, the less easily the mobile device can perceive the detection pipeline.

[0080] Understandably, the reliability quantification value indicates that when the detection pipeline is easily perceived by the mobile device, the distance between the mobile device and the detection pipeline is closer; conversely, the reliability quantification value indicates that when the detection pipeline is less easily perceived by the mobile device, the distance between the mobile device and the detection pipeline is farther.

[0081] In one feasible implementation, the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point can be input into the reliability detection model to obtain the current reliability quantification value output by the reliability detection model.

[0082] In another feasible implementation, the current reliability quantification value can be obtained by weighted summing of the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point.

[0083] In the embodiments of this application, different reliability quantification values ​​correspond to different flight distance constraints, which are used to constrain the lateral and longitudinal distances of the mobile device to the detection pipeline.

[0084] The lateral distance corresponds to the normal direction of the pipe centerline, and the lateral distance represents the distance between the mobile device and the pipe centerline.

[0085] The longitudinal distance corresponds to the tangential direction of the pipeline centerline, and the longitudinal distance represents the distance the mobile device moves along the pipeline centerline.

[0086] S403. Based on the target flight distance constraints corresponding to the current reliability quantification value and the available pipeline length corresponding to the pipeline network, determine the target control command for the mobile device.

[0087] The target flight distance constraint is used to characterize the lateral and longitudinal target flight distances of the mobile device to the detection pipeline. The available pipeline length refers to the length of the effective detection pipeline, which is a detection pipeline without obstructions. The mobile device performs detection within the available pipeline length.

[0088] Target control commands are used to characterize commands that control the mobile device to move autonomously along the detection pipeline.

[0089] The following describes a method for determining the target control commands for mobile devices based on the target flight distance constraints corresponding to the current reliability quantification value and the available pipeline length corresponding to the pipeline network.

[0090] In one feasible implementation, when the available pipeline length is greater than the pipeline length threshold, the target control command of the mobile device is determined based on the target flight distance constraint corresponding to the current reliability quantification value; when the available pipeline length is less than the pipeline length threshold, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value is reduced to obtain the updated target flight distance constraint, thereby determining the target control command of the mobile device.

[0091] In another feasible implementation, based on the available pipeline length, the length of the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value is adjusted to obtain the target flight distance constraint, thereby determining the target control command of the mobile device.

[0092] In this embodiment, two-dimensional perception information can characterize two-dimensional image information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network; three-dimensional perception information can characterize three-dimensional information related to the pipe conditions and surrounding environment of the detected pipes in the pipe network; wherein, the uniqueness determination result of the detected pipes included in the three-dimensional perception information can characterize the probability that each three-dimensional feature point in the three-dimensional perception information can represent the information of the same target detected pipe. Combining two-dimensional and three-dimensional perception information can obtain a more accurate actual perception situation of the detected pipes (that is, the reliability quantification value of the perception capability), and the flight distance constraint of the mobile device determined based on the more accurate actual perception situation is also more accurate. Further, the available pipe length refers to the length of the effective detected pipes, which refers to the unobstructed detected pipes, and the mobile device performs detection within the available pipe length; for unavailable pipes, that is, for other pipes affected by harsh environments or obstructed, the mobile device does not perform detection. When the mobile device moves to other pipes, the flight distance between the mobile device and other pipes is relatively far, thereby avoiding the impact of complex and harsh environments and complex pipe networks on the flight of the mobile device and the collection of perception data. Furthermore, the commands generated based on relatively accurate flight distance constraints and available pipe length are also relatively accurate. Thus, the mobile device can achieve relatively accurate and reliable control during its flight along the detection pipe, thereby improving the safety and continuity of the mobile device's autonomous flight.

[0093] The following provides a detailed explanation of how to acquire two-dimensional and three-dimensional sensing information in step S401.

[0094] S4011. Acquire initial two-dimensional sensing information through the two-dimensional sensing device of the mobile device.

[0095] Among them, the two-dimensional sensing device includes an image acquisition device.

[0096] The initial 2D sensing information includes an image sequence of the pipeline network. This initial 2D sensing information is used to characterize the 2D information of the pipeline itself and the surrounding environment perceived by the 2D sensing device within the pipeline network.

[0097] The following explanation uses a camera as an example of a two-dimensional sensing device. The camera captures images of the detected pipes in the pipe network, acquiring multiple camera image sequences of the detected pipes, which constitute the initial two-dimensional sensing information in this embodiment.

[0098] In some embodiments, initial two-dimensional sensing information can be acquired through the two-dimensional sensing device of a mobile device at a preset period. For example, the preset period can be 1 minute, 1 hour, 1 day, 1 week, 1 month, etc.

[0099] S4012. Acquire initial three-dimensional perception information through the three-dimensional perception device of the mobile device.

[0100] Among them, the three-dimensional sensing equipment includes lidar devices.

[0101] The initial 3D sensing information includes point cloud data of the pipeline network, which contains the coordinate information of 3D feature points. This initial 3D sensing information is used to characterize the 3D information of the pipeline itself being sensed by the 3D sensing equipment within the pipeline network, as well as the 3D information of the surrounding environment.

[0102] The following explanation uses a lidar as an example of a 3D sensing device. The lidar scans the detected pipes in the pipe network using laser light, acquiring the echoes reflected back from the pipes. Based on the time and angle of the echoes, a lidar point cloud corresponding to the detected pipe is obtained, which is the initial 3D sensing information in this embodiment. The lidar point cloud is a set of 3D feature points obtained by the lidar in detecting the pipes. Each 3D feature point includes its 3D coordinates in the lidar coordinate system and information such as echo intensity.

[0103] In some embodiments, initial 3D sensing information can be acquired through the 3D sensing device of a mobile device at a preset period. For example, the preset period can be 1 minute, 1 hour, 1 day, 1 week, 1 month, etc.

[0104] For inspection pipelines requiring more precise detection, the preset cycle can be shorter, for example, acquiring 2D and 3D information from the 2D and 3D sensing devices respectively every 1 millisecond. For inspection pipelines requiring routine detection, the preset cycle can be longer, for example, acquiring 2D and 3D information from the 2D and 3D sensing devices respectively every 5 seconds.

[0105] S4013. Map the initial two-dimensional sensing information and the initial three-dimensional sensing information to the same spatial reference, and determine the pipeline centerline, the two-dimensional sensing range corresponding to the initial two-dimensional sensing information, and the three-dimensional sensing range corresponding to the initial three-dimensional sensing information.

[0106] Mapping initial two-dimensional perception information and initial three-dimensional perception information to the same spatial reference includes three cases.

[0107] In one scenario, the initial two-dimensional perception information and the initial three-dimensional perception information can be mapped to the spatial reference corresponding to the initial three-dimensional perception information.

[0108] In another case, the initial two-dimensional perception information and the initial three-dimensional perception information can be mapped to the spatial reference formed by the plane containing the two-dimensional perception information and the normal to that plane.

[0109] In another scenario, the initial two-dimensional perception information and the initial three-dimensional perception information can be mapped to a preset spatial reference. The preset spatial reference can be the World Geodetic System 1984 (WGS84), the China Geodetic Coordinate System 2000 (CGCS2000), or any unified spatial reference, etc.

[0110] This explanation uses an example where the initial two-dimensional sensing information is a camera image sequence and the initial three-dimensional sensing information is a LiDAR point cloud. Let the camera image sequence be... , Let the frame index be ; let the lidar point cloud be . , For point indexes, establish Let the extrinsic parameters of the camera to the unified spatial reference be... The extrinsic parameters for laser-based radar to achieve a unified spatial reference are: The coordinates in the camera image sequence are arranged according to... Mapping a unified spatial benchmark Next, the coordinates of the lidar point cloud will be set according to... Mapping to a unified spatial benchmark This allows for the alignment of the initial two-dimensional and initial three-dimensional perceptual information within the field of view.

[0111] It is understandable that the initial two-dimensional sensing information includes not only two-dimensional information related to the detection pipeline, but also two-dimensional information unrelated to the detection pipeline. For example, unrelated two-dimensional information includes information about other detection pipelines, environmental information unrelated to the detection pipeline, etc. Unrelated two-dimensional information can easily cause interference, making it difficult to process information related to the detection pipeline.

[0112] The initial 3D sensing information includes not only 3D information related to the detected pipeline, but also 3D information unrelated to the detected pipeline. For example, unrelated 3D information includes information about other detected pipelines and environmental information unrelated to the detected pipeline. Unrelated 3D information can easily cause interference, making it difficult to process information related to the detected pipeline.

[0113] Therefore, the regions related to the detection pipeline in the initial two-dimensional perception information and the initial three-dimensional perception information can be divided, making it easier to perform subsequent analysis and processing of the detection pipeline based on the divided information.

[0114] Before dividing the areas related to the detection pipeline in the initial two-dimensional and initial three-dimensional sensing information, the pipeline centerline of the detection pipeline can be determined based on the initial two-dimensional and initial three-dimensional sensing information.

[0115] In one feasible implementation, the initial two-dimensional perception information and the initial three-dimensional perception information are mapped to the same spatial reference. The process involves acquiring two-dimensional and three-dimensional information. Based on the information corresponding to the inspected pipeline in the two-dimensional information, a set of three-dimensional feature points is discretized and extracted along the pipeline's direction in the three-dimensional information as the initial pipeline inspection result. Let the set of three-dimensional feature points in the initial pipeline inspection result be denoted as . , Represents the i-th 3D feature point, located on the same spatial reference. China. (General) The three-dimensional feature points in the graph are connected in spatial order and fitted to a parametric curve. ,in, This is the arc length parameter along the conductor.

[0116] The pipe centerline is determined by constraining the continuity of curvature to form the pipe centerline trajectory. Specifically, based on a sequence of tangential directions sampled uniformly along the arc length, the variation amplitude of the tangential direction at adjacent sampling points is statistically analyzed, and the curvature is obtained by normalizing the sampling interval. The discrete values ​​are used as constraints for adjustment. The three-dimensional feature points make the curvature The conductor is continuous within the section covered by the initial inspection results and is based on the same spatial reference. The central trajectory of the pipeline is formed. This allows us to obtain the centerline of the pipeline being inspected.

[0117] In another feasible implementation, the initial two-dimensional perception information and the initial three-dimensional perception information are input into the centerline determination model to obtain the centerline of the detection pipeline output by the centerline determination model.

[0118] After obtaining the centerline of the pipeline being inspected, the two-dimensional sensing range corresponding to the initial two-dimensional sensing information and the three-dimensional sensing range corresponding to the initial three-dimensional sensing information are obtained, with the centerline of the pipeline being inspected as the center.

[0119] The 2D sensing range characterizes the region where the detection pipeline is located in the initial 2D sensing information. For example, when the initial 2D sensing information is a sequence of camera images, the 2D sensing range can also be referred to as an image stripe. For the initial 2D sensing information on the same spatial reference... The two-dimensional information below can be obtained through the centerline of the pipeline. The centerline of the two-dimensional pipeline is used as the center. Based on the preset width threshold, the upper and lower limits of the normal direction of the centerline of the two-dimensional pipeline are determined. Thus, the two-dimensional sensing range corresponding to the initial two-dimensional sensing information can be obtained, that is, the area where the pipeline is detected in the initial two-dimensional sensing information.

[0120] The preset width threshold can be an empirically based setting or a set threshold range.

[0121] In some embodiments, the two-dimensional information corresponding to the two-dimensional pipe centerline can be obtained based on the pipe centerline using a camera projection model. Specifically, the pipe centerline trajectory... As input to the camera projection model, the trajectory will be in the pipeline with a unified spatial reference. The three-dimensional feature points are transformed into the camera coordinate system to obtain two-dimensional coordinates based on the camera coordinate system. The pixel positions corresponding to the two-dimensional coordinates are obtained through the camera intrinsic parameter matrix, and the center line pixel trajectory corresponding to the two-dimensional information is obtained, that is, the two-dimensional pipeline centerline in the embodiment of this application.

[0122] Since real lenses have radial and tangential optical distortion, which causes the two-dimensional coordinates to shift, the two-dimensional coordinates can be corrected based on a preset distortion coefficient to obtain the corrected two-dimensional coordinates. Then, the pixel position corresponding to the corrected two-dimensional coordinates can be obtained from the camera intrinsic parameter matrix to obtain the center line pixel trajectory corresponding to the two-dimensional information, thereby making the obtained center line pixel trajectory more accurate.

[0123] The 3D sensing range characterizes the area where the detection pipeline is located in the initial 3D sensing information. For example, when the initial 3D sensing information is a LiDAR point cloud, the 3D sensing range can also be referred to as a point cloud strip.

[0124] For initial three-dimensional sensing information on the same spatial reference The three-dimensional information below can be used to determine the extension radius in the normal direction of the pipe centerline based on a preset radius threshold, with the pipe centerline as the center. This allows us to obtain the three-dimensional perception range corresponding to the initial three-dimensional perception information, that is, the area where the pipe is detected in the initial three-dimensional perception information.

[0125] The preset radius threshold can be an empirically based setting or a set threshold range.

[0126] S4014. Based on the pipeline centerline and the two-dimensional sensing range, the detected pipelines in the image sequence are divided into pipeline segments to obtain the first pipeline segment set.

[0127] The first pipeline segment set includes multiple image sequence segments.

[0128] In this embodiment, the pipeline segment is divided into pipeline segments based on the pipeline centerline to obtain the first pipeline segment set.

[0129] S4015. Based on the pipeline centerline and the three-dimensional perception range, the detected pipeline in the point cloud data is divided into pipeline segments to obtain the second pipeline segment set.

[0130] The second pipeline segment set includes multiple point cloud data segments.

[0131] Based on the pipeline centerline, the pipeline segmentation is performed on the corresponding 3D perception area of ​​the initial 3D perception information to obtain the second pipeline segment set.

[0132] The following section details the methods for obtaining the first pipeline fragment set and the second pipeline fragment set.

[0133] In one feasible implementation, the trajectory of the pipe centerline can be determined based on a preset segment length threshold. The arc length parameter is used to divide the pipeline into segments, resulting in multiple arc length ranges corresponding to each segment. The arc length range of each segment is then mapped to both the two-dimensional and three-dimensional sensing ranges based on a unified spatial reference, thus obtaining a first pipeline segment set and a second pipeline segment set. The preset segment length threshold can be a set value.

[0134] This explanation uses an example where the initial 2D sensing information is a sequence of camera images and the initial 3D sensing information is a LiDAR point cloud. Let the segment length threshold be... In the centerline of the pipeline The upper section is divided into segments according to the arc length parameter, let the first segment be... The arc length range of each segment is To meet The corresponding segment length is not less than And maintain a uniform length except for the last segment. The length of the last segment can be less than... .

[0135] In a unified spatial benchmark Lieutenant General Mapping to the corresponding image stripes and point cloud stripes, thus forming multiple fragment-level image stripes corresponding to different segments. With fragment-level point cloud stripes Let the set of pipeline segments be... , Let k be the k-th pipeline segment. and , , There is a corresponding relationship.

[0136] In another feasible implementation, the two-dimensional pipeline centerline corresponding to the initial two-dimensional sensing information can be determined based on the pipeline centerline. Based on a preset segment length threshold, the two-dimensional pipeline centerline is segmented to obtain a first pipeline segment set of multiple pipeline segments. The length of each pipeline segment is mapped to the three-dimensional sensing range based on a unified spatial reference, thereby obtaining a second pipeline segment set of multiple pipeline segments.

[0137] In this embodiment, each pipeline segment corresponds to the centerline of the conductor, the image sequence segment, and the point cloud data segment, and each pipeline segment has a corresponding start and end point position.

[0138] Figure 5 This is a schematic diagram illustrating a process for obtaining a pipeline segment set, provided as an embodiment of this application. Figure 5 As shown, the method includes: S501. Obtain initial two-dimensional perception information and initial three-dimensional perception information.

[0139] S502. Map the initial two-dimensional perception information and the initial three-dimensional perception information to the same spatial reference, and form the pipeline centerline trajectory by constraining curvature continuity to determine the pipeline centerline.

[0140] S503. Delineate the two-dimensional sensing area and the three-dimensional sensing area with the centerline of the pipeline as the center.

[0141] S504. Based on the arc length parameter of the pipeline centerline, the detected pipeline is divided into pipeline segments to obtain a first pipeline segment set and a second pipeline segment set containing multiple pipeline segments.

[0142] After performing the above steps, a first set of pipe segments and a second set of pipe segments containing multiple pipe segments are obtained respectively.

[0143] S4016. Based on the first pipeline segment set, obtain two-dimensional sensing information.

[0144] Among them, the two-dimensional perception information obtained based on the first pipeline segment set corresponds one-to-one with the image sequence segments in the first pipeline segment set.

[0145] In some embodiments, the aforementioned two-dimensional perception information may include the directional consistency quantization value of the image sequence fragment, the longest continuous length, the number of breaks, and the inflection point marker.

[0146] The orientation consistency quantization value is used to characterize the consistency between the pixel gradient direction and the pipeline centerline direction of an image sequence segment, reflecting the overall situation of the image sequence segment.

[0147] The longest continuous length is used to characterize the longest continuous length within each image sequence segment without any breaks.

[0148] The break count is used to characterize the number of breaks within a segment of an image sequence.

[0149] For directional consistency quantization values, the directional consistency quantization values ​​corresponding to each image sequence segment in the first pipeline segment set can be obtained based on the direction of the pipeline centerline.

[0150] Specifically, based on pixel coordinates in image sequence segments The pixel gradient vector is calculated using the difference operator. and based on Obtain pixel gradient direction With gradient magnitude The pipeline centerline corresponding to the image sequence segment. Projecting onto an image sequence segment yields the tangential direction along the arc length. For each Build to the centerline of the pipeline Arc length parameter of the nearest projection point within an image sequence segment Based on the set of all pixels within each image sequence segment. Obtain the directional consistency quantization value of each image sequence segment.

[0151] The formula for obtaining the directional consistency quantization value is as follows: Equation (1).

[0152] in, For pixel weights, This is the gradient magnitude threshold. The tangential direction of the pipeline centerline. As the direction threshold, This is an indicator function.

[0153] For the longest continuous length and the number of breaks, the longest continuous length and the number of breaks corresponding to each image sequence segment in the first pipeline segment set are obtained based on the wire pixels of the image sequence segment along the pipeline centerline direction.

[0154] Specifically, the k-th image sequence fragment in the first pipeline fragment set Taking the example of the kth segment of the first pipeline, the arc length range is... Construct wire pixel mask When the pixels in an image sequence segment satisfy and hour, This indicates that the pixel is a valid pixel belonging to the detection pipeline; otherwise... This indicates that the pixel does not belong to the detection pipeline and is an invalid pixel. (The step size is not specified.) right Sampling was performed to obtain multiple normal narrow bands perpendicular to the centerline of the pipe segment. The width and step size of the normal narrow bands were determined. Similarly, query the normal narrowband at each sampling location. The value is used to obtain a binary sequence along the pipeline. Based on the binary sequence along the pipeline, continuous segments of the first pipeline segment are identified, and the number of breaks at discontinuities between continuous segments is counted. The longest consecutive length of image sequence segments .

[0155] For inflection point marking, the inflection point marking corresponding to each image sequence segment in the first pipeline segment set can be determined based on preset inflection point determination conditions.

[0156] In some embodiments, if the direction change rate of an image sequence segment is greater than or equal to a preset direction change rate threshold, the occlusion segment of each image sequence segment in the first pipeline segment set can be determined. Based on the occlusion segment, the occlusion arc length of each image sequence segment in the first pipeline segment set can be obtained, thereby determining the inflection point marker corresponding to each image sequence segment in the first pipeline segment set based on the occlusion arc length and the image sequence segment.

[0157] Specifically, the k-th image sequence fragment in the first pipeline fragment set Let's take an example to illustrate. The arc length range of the k-th image sequence segment is... Based on the pipeline centerline Centered on normal bandwidth Inner pixel gradient direction Local aggregation is performed, and the angle is calculated by weighted averaging of unit direction vectors. A direction profile is then established along the arc length. Through finite differences in Calculate the rate of change of direction. Set the rate of change of direction to be greater than or equal to a preset threshold. The segments are merged to obtain the occluded segments, and the total arc length of the occluded segments is accumulated. To obtain image sequence fragments Corresponding inflection point marker .

[0158] Among them, obtaining the inflection point marker The formula is: Equation (2).

[0159] Based on the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers of each image sequence segment of the first pipeline segment set obtained above, the two-dimensional perception information corresponding to the first pipeline segment set is determined.

[0160] It should be noted that the two-dimensional sensing information can be obtained by splicing together the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers corresponding to each image sequence segment based on the spatial order of each image sequence segment. Alternatively, the two-dimensional sensing information can be obtained by filling the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers corresponding to each image sequence segment into a preset fixed format of two-dimensional sensing information based on the spatial order of the image sequence segments.

[0161] For example, when the initial two-dimensional perception information is a sequence of camera images, the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers of the image sequence segments constitute image-side evidence. The image-side evidence of each image sequence segment is spliced ​​together in spatial order to obtain two-dimensional perception information, which can also be called an image-side evidence vector.

[0162] In other embodiments, the two-dimensional sensing information also includes the low quantile proportion of the orientation consistency quantization value. The low quantile proportion is used to characterize local defects in image sequence segments.

[0163] Specifically, the k-th image sequence fragment in the first pipeline fragment set Let's take an example to illustrate. The arc length range of the k-th image sequence segment is... Will Window length calculated by low quantile The data is divided into several equal-length sub-intervals. For each sub-interval, the directional consistency quantization value and weight are calculated for its pixel set. Calculate the locally consistent proportion sequence. Using the low quantile level... Extract quantile values ​​from this sequence, denoted as the low quantile proportion. . With the k-th image sequence fragment Maintain correspondence.

[0164] In some embodiments, occluded segments of an image sequence can be removed, and the directional consistency quantization value, low quantile ratio, longest continuous length, and number of breaks after removing the occluded segments can be obtained to obtain two-dimensional perception information after removing the occluded segments, thereby making the information related to the detection pipeline in the two-dimensional perception information more accurate.

[0165] Figure 6 This is a schematic diagram illustrating a process for obtaining two-dimensional sensing information corresponding to a first pipeline segment set, provided as an embodiment of this application. Figure 6 The above includes: S601. Based on the direction of the pipeline centerline, obtain the directional consistency quantification index and low quantile ratio of each image sequence segment in the first pipeline segment set.

[0166] S602. Obtain the longest continuous length and the number of breaks corresponding to the image sequence segment along the centerline direction of the pipeline.

[0167] S603. Based on the preset inflection point determination conditions, determine the occlusion segment and inflection point marker corresponding to each image sequence segment of the first pipeline segment set.

[0168] S604. Remove the obstructed sections from the first pipeline segment set. Based on the first pipeline segment set after removing the obstructed sections, return to step S601 above to recalculate the directional consistency quantification index, low quantile ratio, longest continuous length, number of breaks, and inflection point marker.

[0169] S605. Based on the directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers of each image sequence segment in the first pipeline segment set, determine the two-dimensional perception information corresponding to the first pipeline segment set.

[0170] In this embodiment, structural misalignment caused by transverse tower components or crossings within the two-dimensional sensing range is constrained by occlusion inflection point markings. This ensures that the directional consistency index and continuity index only cover the continuous usable section along the conductor, forming two-dimensional sensing information for autonomous flight determination along the conductor. This makes the target control commands obtained based on the two-dimensional sensing information more accurate, and further, makes it safer for moving objects to perform autonomous flight along the conductor based on target control commands.

[0171] After performing the above steps, the two-dimensional sensing information corresponding to the first pipeline segment set was obtained.

[0172] S4017. Based on the second pipeline segment set, obtain three-dimensional perception information.

[0173] Among them, the three-dimensional perception information obtained based on the second pipeline segment set corresponds one-to-one with the point cloud data segments in the second pipeline segment set.

[0174] In some embodiments, the three-dimensional perception information includes a first distribution uniformity quantization value, an echo stability quantization value, and a first fine-tuning stability quantization value for each point cloud data segment in the second pipeline segment set.

[0175] The first distribution uniformity quantization value is used to characterize the distribution of three-dimensional feature points in a point cloud data segment.

[0176] The echo stability quantization value is used to reflect the impact of weather scattering on point cloud reliability.

[0177] The first fine-tuning stability quantization value is used to reflect the distribution of three-dimensional feature points in the three-dimensional perception area of ​​the point cloud data segment after width fine-tuning.

[0178] The first distribution uniformity quantization value can be obtained by calculating the proportion of three-dimensional feature points per unit length of the point cloud data segment.

[0179] Specifically, the k-th point cloud data fragment in the second pipeline fragment set Let's take an example to illustrate. The arc length range of the k-th point cloud data segment is... Will Standard for length measurement Divide the window into equal-length windows, and let the first window be... Each window is This yields the normal narrow band perpendicular to the pipe centerline corresponding to the m-th window, where the width of the normal narrow band is the same as the length of the window. Corresponding normal narrow-band statistical fall-in The points are recorded as the points that can be counted directly. ,by As a proportion of the support points per unit length. Forming a sequence within the segment. And calculate the mean of the sequence. with sample standard deviation Through the coefficient of variation Characterize the uniformity of the length direction and quantify the uniformity of the first distribution. Defined as the complement of this coefficient of variation.

[0180] In one specific implementation, the quantization value of the first distribution uniformity is calculated. The formula is: Equation (3).

[0181] Among them, the length measurement standard It can be a set value.

[0182] The echo stability quantization value can be obtained by measuring the echo intensity variation amplitude and outlier ratio of each three-dimensional feature point in the point cloud data segment.

[0183] Outlier ratio is the proportion of 3D feature points that are not in a point cloud data segment.

[0184] Specifically, the k-th point cloud data fragment in the second pipeline fragment set Let's take an example to illustrate. The arc length range of the k-th point cloud data segment is... Let the echo intensity of each three-dimensional feature point be... ,exist Internal collection Calculate the mean of point cloud data segments with sample standard deviation ,by As the amplitude of echo intensity change, the amplitude of echo intensity change Based on the threshold of echo intensity variation Proportion normalization, truncation to The normalized amplitude is obtained. The threshold for the amplitude of echo intensity variation can be a fixed value based on experience.

[0185] Let the outlier magnitude threshold be... ,when The time interval is recorded as an outlier, and the cumulative outlier percentage is calculated. The proportion of outliers Outlier threshold Normalize the proportions and truncate to The normalized outlier ratio was obtained. The outlier threshold can be an empirically based fixed value.

[0186] Based on preset weighting coefficients and normalized amplitude and normalized outlier ratio Perform a weighted summation. Subtract this sum as the quantization value for echo stability. .

[0187] In one specific implementation, the echo stability quantization value The calculation formula is: Equation (4).

[0188] Among them, the preset weight coefficients and It can be a set value.

[0189] For the first fine-tuning stability quantization value, the preset radius threshold corresponding to the three-dimensional perception area can be increased or decreased based on the preset fine-tuning amplitude, and the proportion of three-dimensional feature points per unit length of the point cloud data segment after the increase and decrease can be calculated respectively to obtain the first fine-tuning stability quantization value.

[0190] Specifically, the k-th point cloud data fragment in the second pipeline fragment set Let's take an example to illustrate. The arc length range of the k-th point cloud data segment is... Let the width adjustment range be... ,exist The interior is respectively with and Two radius thresholds are used to construct a first candidate sensing range and a second candidate sensing range. Based on the above method for obtaining the quantization value of the first distribution uniformity, two sets of unit length support point ratio sequences corresponding to the first candidate sensing range and the second candidate sensing range are obtained. and .

[0191] The arc length range of the k-th point cloud data segment is Will Standard for length measurement Divide the window into equal-length windows, and let the first window be... Each window is This yields the normal narrow band perpendicular to the pipe centerline corresponding to the m-th window, with the width of the normal narrow band being the same as the length of the window. For the first candidate sensing range and the second candidate sensing range... Corresponding normal narrowband calculation In the point cloud data fragment The maximum value is used as the first fine-tuning stability quantization value. .

[0192] Based on the first distribution uniformity quantization value, echo stability quantization value, and first fine-tuning stability quantization value of each point cloud data segment in the second pipeline segment set, the three-dimensional perception information corresponding to the second pipeline segment set is determined.

[0193] It should be noted that the 3D perception information is based on the spatial order of each point cloud data segment, and is formed by splicing together the first distribution uniformity quantization value, echo stability quantization value, and first fine-tuning stability quantization value corresponding to each point cloud data segment.

[0194] For example, when the initial three-dimensional perception information is a lidar point cloud, the first distribution uniformity quantization value, the echo stability quantization value, and the first fine-tuning stability quantization value of the point cloud data segment constitute the laser measurement evidence. The laser measurement evidence of each point cloud data segment is spliced ​​in spatial order to obtain the three-dimensional perception information, which can also be called the laser side evidence vector.

[0195] In other embodiments, the 3D perception information also includes a unique quantification value.

[0196] The uniqueness quantification value is used to characterize the probability that each three-dimensional feature point in the second pipeline set can represent the information of the same target detection pipeline.

[0197] The unique quantization value can be obtained based on the first fine-tuning stability quantization value, the second distribution uniformity quantization value, and the second fine-tuning stability quantization value. Based on a preset offset pipeline centerline parallel to the pipeline centerline, the second distribution uniformity quantization value and the second fine-tuning stability quantization value of each point cloud data segment in the offset second pipeline segment set are obtained. Based on the first distribution uniformity quantization value, the first fine-tuning stability quantization value, the second distribution uniformity value, and the second fine-tuning stability quantization value, the unique quantization value of each point cloud data segment in the second pipeline segment set is obtained through a preset unique quantization formula.

[0198] Specifically, the k-th point cloud data fragment in the second pipeline fragment set Taking the example of the k-th point cloud data segment, the arc length range is... Let the parallel offset distance be... With the centerline of the pipeline The normal direction is in Built-in parallel offset control line Its arc length range is Consistent. Based on parallel offset reference lines. Calculate the second uniform quantization value of each point cloud data fragment in the offset second pipeline fragment set. And the second fine-tuning stability quantization value

[0199] The control results can be characterized by the difference between the first uniform quantization value and the second uniform quantization value, and by the difference between the first fine-tuning stability quantization value and the second fine-tuning stability quantization value. Let the control difference threshold be... Define the ratio difference of support points per unit length. The difference between the adjustment range and the control range Therefore, a unique quantization value can be obtained based on a preset unique quantization formula. .

[0200] The uniqueness quantification formula is as follows: Equation (5).

[0201] in, , , These are preset weighting coefficients; This is the first fine-tuning stability quantization value.

[0202] Based on the first distribution uniformity quantization value, echo stability quantization value, first fine-tuning stability quantization value, and uniqueness quantization value of each point cloud data segment in the second pipeline segment set, the three-dimensional perception information corresponding to the second pipeline segment set is determined.

[0203] In this embodiment, the uniqueness determination result of each point cloud data segment in the second pipeline segment set can be obtained based on the uniqueness quantization value and the uniqueness quantization threshold. The uniqueness determination result is used to characterize the reliability of the point cloud data segments.

[0204] If the uniqueness quantization value is greater than or equal to the uniqueness quantization threshold, the uniqueness determination result for each point cloud data segment in the second pipeline segment set is "yes". A uniqueness determination result of "yes" indicates that the reliability of each point cloud data segment in the second pipeline segment set is high.

[0205] If the uniqueness quantization value is less than the uniqueness quantization threshold, the uniqueness determination result for each point cloud data segment in the second pipeline segment set is negative. A negative uniqueness determination result indicates that the reliability of each point cloud data segment in the second pipeline segment set is low.

[0206] Figure 7 This is a schematic diagram illustrating a process for obtaining three-dimensional perception information corresponding to a second pipeline segment set, provided as an embodiment of this application. Figure 7 The above includes: S701, Obtain the second pipeline fragment set.

[0207] S702. Based on the direction of the pipeline centerline, determine the number of three-dimensional feature points of each point cloud data segment in the second pipeline segment set within a preset length, and obtain the first distribution uniformity quantization value of each point cloud data segment in the second pipeline segment set.

[0208] S703. Based on the echo intensity of each point cloud data segment in the second pipeline segment set, determine the echo stability quantization value of each point cloud data segment in the second pipeline segment set.

[0209] S704. Based on the preset fine-tuning amplitude and the three-dimensional perception range of the point cloud data segment, obtain the first fine-tuning stability quantization value of each point cloud data segment in the second pipeline segment set.

[0210] S705. In accordance with the first distribution uniformity quantization value in step S702 and the first fine-tuning stability quantization value in step S704, based on a preset offset pipeline centerline parallel to the pipeline centerline, obtain the second distribution uniformity quantization value and the second fine-tuning stability quantization value of each point cloud data segment in the offset second pipeline segment set.

[0211] S706. Based on the first distribution uniformity quantization value, the first fine-tuning stability quantization value, the second distribution uniformity quantization value, and the second fine-tuning stability quantization value, the uniqueness determination result of each point cloud data segment in the second pipeline segment set is obtained through a preset uniqueness quantization formula.

[0212] S707. Based on the first distribution uniformity quantization value, the echo stability quantization value, the first fine-tuning stability quantization value, and the uniqueness determination result, determine the three-dimensional perception information corresponding to the second pipeline segment set.

[0213] In this embodiment, the three-dimensional feature points within the three-dimensional perception range are judged to determine whether the three-dimensional feature points originate from a single conductor, and false supports caused by scattering from nearby structures or weather are excluded. This makes the obtained three-dimensional perception information more practical and the target control commands obtained based on the three-dimensional perception information more accurate. Furthermore, it makes it safer for moving objects to fly autonomously along the line based on target control commands.

[0214] The method for determining the current reliability quantification value of the mobile device's sensing capability in step S402 is described in detail below.

[0215] S4021. Obtain the projection overlap ratio based on two-dimensional and three-dimensional perception information.

[0216] Among them, the projection overlap ratio represents the degree of overlap between the two-dimensional perception information and the three-dimensional perception information corresponding to each segment in the pipeline segment set.

[0217] In one feasible implementation, the 3D sensing information includes the pipe centerline and the 3D sensing range of the detected pipe, while the 2D sensing information includes the 2D sensing range. Under the same spatial reference, based on the pipe centerline and the 3D sensing range corresponding to the 3D sensing information, the 3D traverse is projected onto the 2D sensing range corresponding to the 2D sensing information to obtain 2D projection information. Based on the 2D projection information and the 2D sensing information, the coincident traverse is determined, and the projection overlap ratio is obtained.

[0218] Specifically, this will be illustrated using an example where the two-dimensional perception range is represented by image stripes and the three-dimensional perception range by point cloud stripes. Let the centerline of the pipeline be... , Let be the arc length parameter along the line, and let the first be the arc length parameter. The pipeline segment is Its arc length range is Let the fragment-level point cloud stripes be... Centerline and pipe centerline Overlapping, let the segment-level image stripe be... Let the camera image sequence be... Indexed by fragment With arc length parameter Constructing a spatial sequence, making , , and Maintain correspondence, and As the start and end points of the pipeline segment.

[0219] First, based on the laser-side evidence vector pair A segment-level 3D guideline is obtained by weighted fitting of the inner 3D feature points. Let the 3D feature points be... , To unify the coordinates of three-dimensional feature points under a unified coordinate datum, let the echo intensity of the three-dimensional feature points be... ,by For reference, using step size right Sampling was performed to obtain multiple normal narrow bands perpendicular to the centerline of the pipe segment. The width and step size of the normal narrow bands were determined. The same applies, let the sampling sequence be... .

[0220] For each exist Collect neighboring points within the normal narrow band, and denote the neighboring points as {p}. j}, calculate the distance from the nearest neighbor point to the nearest neighbor point. The normal distance is denoted as Using a preset radius threshold Will Normalization Cut off to The average echo intensity of the segment. Will Cut off to The result is determined by uniqueness. As a fragment-level gating, when If no, reset the weight to zero; when In this case, the weight is obtained by multiplying the two items mentioned above. .

[0221] Through the neighboring point {p j The weights of} and neighboring points Weighted centroid calculation is performed to obtain Using cubic interpolation A continuous curve is formed on the top, resulting in a three-dimensional traverse. Among them, three-dimensional guide wires and Maintain fragment-level correspondences. Translate the 3D guide wires. By projecting the spatial reference of the pipe centerline and the image stripes onto the camera image sequence, the overlap ratio of the image stripes and the point cloud stripes is obtained. Let the camera projection model be... Under a unified coordinate reference The pixel trajectory is obtained by point-by-point projection and discretized into a set of projected pixels. Let the pixel mask of the image side guide be... , its in The internal structure is generated by determining directional consistency and gradient magnitude threshold, with... Calculate the projection overlap ratio at the fragment level, denoted as . and with Maintain correspondence.

[0222] In another feasible implementation, two-dimensional and three-dimensional sensing information can be input into the projection overlap ratio model. By converting the three-dimensional sensing information into the plane where the two-dimensional sensing information is located, the projection overlap ratio can be obtained by obtaining the probability of the projected three-dimensional sensing information in the plane where the two-dimensional sensing information is located.

[0223] S4022. Input the projection overlap ratio, two-dimensional perception information, three-dimensional perception information and the uniqueness determination results of each three-dimensional feature point into the perception reliability quantification model, and output the current reliability quantification value of the mobile device's perception capability.

[0224] In one feasible implementation, the perceptual reliability quantification model includes a two-dimensional scoring network, a three-dimensional scoring network, a projection overlap ratio scoring network, and a fusion scoring network. Two-dimensional perceptual information is input into the two-dimensional scoring network to obtain a two-dimensional score. Three-dimensional perceptual information is input into the three-dimensional scoring network to obtain a three-dimensional score. The projection overlap ratio is input into the projection overlap ratio scoring network to obtain a projection score corresponding to the projection overlap ratio. The two-dimensional score, three-dimensional score, overlap score, and the uniqueness determination results of each three-dimensional feature point are input into the fusion scoring network to obtain the current reliability quantification value of the mobile device's perceptual capability.

[0225] In another feasible implementation, the perceived reliability quantification model includes a two-dimensional feature extraction network, a three-dimensional feature extraction network, and a fusion network. Two-dimensional perceived information is input into the two-dimensional feature extraction network to obtain the corresponding two-dimensional feature embedding vector. Similarly, three-dimensional perceived information is input into the three-dimensional feature extraction network to obtain the corresponding three-dimensional feature embedding vector. The two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination results of each three-dimensional feature point are then input into the fusion network to obtain a fused feature vector. Based on this fused feature vector, the current reliability quantification value of the mobile device's perception capability is determined.

[0226] Among them, the perceptual reliability quantification model can be obtained by splicing and fusing two-dimensional feature embedding vectors, three-dimensional feature embedding vectors, projection overlap ratios, and the uniqueness determination results of each three-dimensional feature point.

[0227] The perceptual reliability quantification model can also obtain a fused feature vector by calculating the difference vector between the two-dimensional feature embedding vector and the three-dimensional feature embedding vector, as well as the product vector between the two-dimensional feature embedding vector and the three-dimensional feature embedding vector, and then splicing and fusing the difference vector, the product vector, the two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination results of each three-dimensional feature point.

[0228] In some embodiments, the perceived reliability quantization model further includes a decision network. The fused feature vector is input to the decision network to obtain the current reliability quantization value corresponding to the mobile device's perception capability and the anomaly type corresponding to the current reliability quantization value.

[0229] The decision network is used to obtain the reliability quantification value corresponding to the detection pipeline, as well as the anomaly type corresponding to the reliability quantification value.

[0230] In this embodiment, the anomaly types include two-dimensional perception information anomalies, three-dimensional perception information anomalies, and overlap anomalies.

[0231] Two-dimensional sensing information anomalies are used to characterize the insufficient two-dimensional information in the two-dimensional sensing information expression of the detection pipeline and the surrounding environment.

[0232] The anomaly in 3D sensing information is due to insufficient 3D information used to characterize the condition of the detection pipeline and the surrounding environment.

[0233] Overlap anomalies are used to characterize the insufficient information expressing the condition of the detection pipeline and the surrounding environment after the two-dimensional and three-dimensional sensing information overlaps.

[0234] In this embodiment, the reliability quantification value and anomaly type obtained based on multi-dimensional evidence such as two-dimensional perception information, three-dimensional perception information, projection overlap ratio, and uniqueness determination results can avoid the impact of external environmental factors such as multiple parallel wires, occlusion, and weather, thereby improving the practicality and reliability of the reliability quantification value.

[0235] Figure 8 This is a schematic diagram illustrating a process for obtaining reliability quantification values ​​based on a perceived reliability quantification model, provided in an embodiment of this application.

[0236] S801. Input the two-dimensional perception information, three-dimensional perception information, projection overlap ratio, and uniqueness determination result into the perception reliability quantification model.

[0237] S802. Based on the two-dimensional feature extraction network and the three-dimensional feature extraction network, obtain the two-dimensional feature embedding vector corresponding to the two-dimensional perception information and the three-dimensional feature embedding vector corresponding to the three-dimensional perception information, respectively.

[0238] S803. Input the two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination result of each three-dimensional feature point into the fusion network to obtain the fusion feature vector.

[0239] S804. Input the fused feature vector into the decision network to obtain the current reliability quantization value of the mobile device's perception capability and the anomaly type corresponding to the current reliability quantization value.

[0240] The perceptual reliability quantification model includes a two-dimensional extraction network, a three-dimensional feature extraction network, a fusion network, and a decision network.

[0241] For example, Figure 9 This is a network architecture diagram of a perceptual reliability quantification model provided in an embodiment of this application.

[0242] like Figure 9 As shown, the perceptual reliability quantification model includes an input layer, an intermediate layer, and an output layer.

[0243] Specifically, the input layer includes: two-dimensional perception information, three-dimensional perception information, projection overlap ratio, and uniqueness determination result. The two-dimensional perception information includes directional consistency quantization index, low quantile ratio, longest continuous length, number of breaks, and inflection point markers. The three-dimensional perception information includes first distribution uniformity quantization value, echo stability quantization value, and first fine-tuning stability quantization value.

[0244] The intermediate layer includes a two-dimensional feature extraction network, a three-dimensional feature extraction network, and a fusion network.

[0245] Two-dimensional feature extraction networks can output two-dimensional feature embedding vectors E_img based on two-dimensional perceptual information, while three-dimensional feature extraction networks can output three-dimensional feature embedding vectors E_lidar based on three-dimensional perceptual information. Both two-dimensional and three-dimensional feature extraction networks can be multilayer perceptrons (MLPs). The network structure of a multilayer perceptron can be FC-ReLU-FC, consisting of two fully connected layers (FC) and a rectified linear unit (ReLU) activation function. The first fully connected layer receives the input data and maps it to a new, higher-dimensional feature space through linear transformation; the ReLU activation function enables the network to learn complex nonlinear relationships; and the second fully connected layer maps the features from the hidden layers to the final output space.

[0246] The fusion network can concatenate the two-dimensional feature embedding vector E_img, the three-dimensional feature embedding vector E_lidar, the projection overlap ratio r_proj, and u. Based on the two-dimensional feature embedding vector E_imgy and the three-dimensional feature embedding vector E_lidar, the difference vector Δ and the product vector ∏ are obtained, thereby obtaining the fusion feature F=[concatenation, Δ, ∏, r_proj, u].

[0247] The output layer includes a decision network, which, based on fused features, obtains the current reliability quantification value of the mobile device's perception capability and the corresponding anomaly type. The decision network structure can be FC×2-ReLU-Softmax. The first fully connected layer maps the input data to a higher-dimensional hidden feature space; the ReLU activation function enables the network to learn complex nonlinear relationships; the second fully connected layer maps the hidden layer features to class scores; and Softmax converts the class scores into a probability distribution. In scenarios with uniqueness constraints, reliability quantification values ​​with a negative uniqueness determination result correspond to a low level.

[0248] In this embodiment, the lower the current reliability quantification value of the mobile device's sensing capability, the greater the flight distance indicated by the flight distance constraint.

[0249] Understandably, the lower the current reliability quantification value of the mobile device's sensing capability, the lower the reliability of the detection pipeline being accurately sensed. In other words, the two-dimensional and three-dimensional sensing information related to the detection pipeline represents the complex situation of the detection pipeline and the surrounding environment, which is not conducive to the mobile device approaching the detection pipeline for detection. Therefore, the greater the distance between the mobile device and the detection pipeline, the smaller the distance the mobile device moves along the detection pipeline. That is, the greater the flight distance indicated by the flight distance constraint.

[0250] In one feasible implementation, the flight distance constraints of the mobile device in the corresponding pipeline segment can be determined based on the current reliability quantification value and the preset reliability quantification threshold.

[0251] A preset reliability quantization threshold is used to characterize the reliability of the detected pipeline. If the current reliability quantization value is greater than the preset reliability quantization threshold, it indicates that the mobile device can accurately detect the detected pipeline; if the current reliability quantization value is less than the preset reliability quantization threshold, it indicates that the mobile device can detect the detected pipeline, but the accuracy of the detection is low.

[0252] If the current reliability quantification value is greater than the preset reliability quantification threshold, the flight distance constraint condition of the mobile device in the corresponding pipeline segment is determined to be short-distance movement. Short-distance movement means that the distance between the mobile device and the centerline of the detected pipeline is greater than the radius of the detected pipeline and less than the movement threshold. If the current reliability quantification value is less than or equal to the preset reliability quantification threshold, the flight distance constraint condition of the mobile device in the corresponding pipeline segment is determined to be long-distance movement. Long-distance movement means that the distance between the mobile device and the centerline of the detected pipeline is greater than the movement threshold and less than the preset radius threshold corresponding to the three-dimensional perception range.

[0253] In one feasible implementation, the flight distance constraints of the mobile device in the corresponding pipeline segment can be determined based on the numerical level corresponding to the current reliability quantification value and the magnitude relationship between the first threshold and the second threshold. The flight distance constraints include corresponding lateral distance constraints and longitudinal distance constraints.

[0254] The current reliability quantification value can be obtained based on a fixed mapping relationship to determine the corresponding numerical level. For example, let the numerical level be... Set the current reliability quantization value to a fixed mapping. Among them, low corresponds , corresponding to High correspondence The numerical rating is used to characterize the reliability of the pipeline being accurately detected.

[0255] The first threshold is used to characterize the upper limit of the reliability level at which the detection pipeline can be perceived. If the numerical level corresponding to the current reliability quantification value is greater than the first threshold, it indicates that the mobile device can accurately perceive the detection pipeline; if the numerical level corresponding to the current reliability quantification value is less than the first threshold, it indicates that the mobile device can perceive the detection pipeline relatively accurately.

[0256] The second threshold is used to characterize the lower limit of the reliability at which the detected pipeline can be perceived. If the numerical level corresponding to the current reliability quantification value is greater than the second threshold, it indicates that the mobile device can perceive the detected pipeline relatively accurately; if the numerical level corresponding to the current reliability quantification value is less than the second threshold, it indicates that the mobile device can perceive the detected pipeline, but the accuracy of perception is low.

[0257] The lateral distance constraint is aligned with the normal to the centerline of the pipe. The lateral distance constraint is used to limit the distance between the mobile device and the centerline of the pipe. In other words, the mobile device moves perpendicular to the extension direction of the detected pipe.

[0258] The longitudinal distance constraint is aligned with the tangent direction of the pipe centerline. The longitudinal distance constraint is used to limit the distance the mobile device moves along the pipe centerline, that is, the mobile device moves along the extension direction of the detected pipe.

[0259] Specifically, if the current reliability quantification value is greater than the first threshold, the flight mode of the mobile device in the corresponding pipeline segment is determined to be the line-following mode. The line-following mode is the flight mode of the mobile device that is closest to the detection pipeline when it flies autonomously along the detection pipeline.

[0260] If the current reliability quantification value is less than the first threshold and greater than the second threshold, the flight mode of the mobile device in the corresponding pipeline segment is determined to be cautious mode. Cautious mode is the flight mode in which the mobile device flies closer to the detection pipeline when flying autonomously along the detection pipeline. Specifically, the distance between the cautious mode and the detection pipeline is greater than the distance between the cautious mode and the detection pipeline.

[0261] If the current reliability quantification value is less than the second threshold, the flight mode of the mobile device in the corresponding pipeline segment is determined to be the safety-maintaining mode. The safety-maintaining mode is the flight mode in which the mobile device flies at a greater distance from the detection pipeline while autonomously flying along the detection pipeline. Specifically, the distance between the safety-maintaining mode (cautionary mode) and the detection pipeline is greater than the distance between the safety-maintaining mode and the detection pipeline.

[0262] In one embodiment, the flight distance constraints corresponding to the following mode include a first lateral distance threshold and a first longitudinal distance threshold; the flight distance constraints corresponding to the cautious mode include a second lateral distance threshold and a second longitudinal distance threshold; and the flight distance constraints corresponding to the safe-keeping mode include a third lateral distance threshold and a third longitudinal distance threshold.

[0263] It is understandable that the first horizontal distance threshold is less than the second and third horizontal distance thresholds, and the second horizontal distance threshold is less than the third horizontal distance threshold; the first vertical distance threshold is less than the second and third vertical distance thresholds, and the second vertical distance threshold is less than the third vertical distance threshold.

[0264] In the embodiments of this application, the first lateral distance threshold, the second lateral distance threshold, the third lateral distance threshold, the first longitudinal distance threshold, the second longitudinal distance threshold, and the third longitudinal distance threshold can be set values ​​based on experience.

[0265] For example, let the current reliability quantification value be... The corresponding numerical level is ,Will Set to fixed mapping Among them, low corresponds , corresponding to High correspondence Let the first threshold be... The second threshold is It is understandable that satisfying Let the k-th pipe segment be... The corresponding flight mode is Then when Time setting In follow-up mode, when Time setting For cautious mode, when Time setting For safety, maintain mode, and, flight mode. With the kth pipe segment Maintain correspondence and record. and The mapping relationship is used for the generation of subsequent control commands.

[0266] Let the lateral distance threshold corresponding to the lateral distance constraint be... The vertical distance threshold corresponding to the vertical distance constraint is Let the first lateral distance constraint corresponding to the following line mode be... The first longitudinal distance constraint is The second lateral distance constraint corresponding to the cautious mode is The second longitudinal distance constraint is The third lateral distance constraint corresponding to the safety-maintaining mode is The third longitudinal distance constraint is .

[0267] Therefore, when When in line mode , .

[0268] when When in cautious mode , .

[0269] when When in safe mode , .

[0270] At this point, the flight distance constraint condition corresponding to the pipeline segment for The following provides a detailed description of step S403.

[0271] Based on the target flight distance constraints corresponding to the current reliability quantification value and the available pipeline length corresponding to the pipeline network, the target control command for the mobile device is determined. For example, let the target control command be... The available pipe length is ,at this time, The corresponding flight distance constraint is The corresponding available test pipe length is .

[0272] In some embodiments, the flight distance constraint can be updated based on the recovery flag corresponding to the pipeline segment. Specifically, if the available pipeline length is greater than or equal to a preset length threshold, and the current reliability quantification value is within a first preset range, the corresponding recovery flag is "yes," indicating that the pipeline segment satisfies the recovery flag, and the target flight distance constraint corresponding to the current reliability quantification value of the pipeline segment remains unchanged.

[0273] If the available pipeline length is less than a preset length threshold, and the current reliability quantification value is within a first preset range, the corresponding recovery flag is "No," indicating that the pipeline segment does not meet the recovery flag. The flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value of the pipeline segment is then reduced, resulting in an updated target flight distance constraint. The preset length threshold can be a set value.

[0274] In this embodiment, the reliability quantification value can be within a first preset range, indicating that the current reliability quantification value is less than a first threshold, and the flight mode of the pipeline segment corresponding to the current reliability quantification value is either a cautious mode or a safety-maintaining mode.

[0275] The reliability quantification value can also be within the second preset range, indicating that the current reliability quantification value is greater than the first threshold, and the flight mode of the pipeline segment corresponding to the current reliability quantification value is the line-following mode.

[0276] In this embodiment, if the current reliability quantification value is within a second preset range, the projection overlap ratio is greater than or equal to a preset overlap threshold, and the uniqueness determination result is yes, then the available pipeline length corresponding to the pipeline network is determined. The preset overlap threshold can be a set value.

[0277] The flight distance indicated in the target flight distance constraint corresponding to the reduction of the current reliability quantification value can be obtained through the following method.

[0278] In one feasible implementation, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value can be reduced by a preset reduction distance.

[0279] In another feasible implementation, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value can be reduced by adjusting the start and end points of the corresponding arc length range of the pipeline segment through a preset reduction factor.

[0280] For example, let the reduction factor be... Let be a dimensionless ratio, and let the range of the effective arc length be . .when If not, the reduced arc length range for: Equation (6).

[0281] Thus, the updated flight distance constraints are obtained as follows: .

[0282] In other embodiments, before the pipeline is marked for restoration, the adjustment direction and adjustment magnitude corresponding to the flight distance constraint can be determined based on the anomaly type of the current reliability quantification value corresponding to the pipeline segment.

[0283] Specifically, based on the anomaly type corresponding to the current reliability quantification value, the adjustment direction and adjustment range corresponding to the flight distance constraint are determined. The flight distance constraint is then adjusted based on the adjustment direction and adjustment range to obtain the updated target flight distance constraint.

[0284] In one feasible implementation, the anomaly types include two-dimensional perception information anomalies, three-dimensional perception information anomalies, and overlap anomalies, and the adjustment directions include horizontal and vertical directions. If the anomaly type corresponding to the current reliability quantification value is a two-dimensional perception information anomaly, the adjustment direction corresponding to the two-dimensional perception information anomaly is determined to be vertical, and the adjustment magnitude corresponding to the vertical direction is a preset first vertical magnitude.

[0285] If the anomaly type corresponding to the current reliability quantification value is a three-dimensional perception information anomaly, determine that the adjustment direction corresponding to the three-dimensional perception information anomaly is horizontal, and the adjustment range corresponding to the horizontal is the preset second horizontal range.

[0286] If the anomaly type corresponding to the current reliability quantification value is the overlapping anomaly, the adjustment direction corresponding to the overlapping anomaly is determined to be horizontal and vertical, the adjustment range corresponding to the horizontal is a preset third horizontal range, and the adjustment range corresponding to the vertical is a preset third vertical range.

[0287] This explanation uses two-dimensional sensing information as the image-side evidence vector and three-dimensional sensing information as the laser-side evidence vector as an example. Let the failure cause type be... The set of values ​​is , respectively representing two-dimensional perception information anomalies, three-dimensional perception information anomalies, and overlap anomalies.

[0288] Let the current lateral distance threshold of the k-th pipe segment be... Compared to the current vertical distance threshold The lateral distance threshold corresponding to the k-th pipe segment is With longitudinal distance threshold Let the first horizontal amplitude be The first longitudinal amplitude is The second horizontal amplitude is The second longitudinal amplitude is The third horizontal amplitude is Third longitudinal amplitude All of these are length measurements.

[0289] Therefore, when hour, , .

[0290] when hour, , .

[0291] when hour, , .

[0292] In some embodiments, after the pipeline is marked for restoration, it is determined whether to perform restoration control based on the projection overlap ratio of two-dimensional and three-dimensional sensing information and a preset overlap threshold.

[0293] The recovery control is used to restore the current flight distance constraint to the preset flight distance constraint when the projection overlap ratio is greater than or equal to the preset overlap threshold.

[0294] The preset flight distance constraint is the initial flight distance constraint corresponding to the pipeline segment. That is, the flight distance constraint is not affected by the anomaly type of the reliability quantification value or the adjustment of the recovery flag.

[0295] Specifically, if the projection overlap ratio is greater than or equal to the preset overlap threshold, the preset flight distance constraint is used as the target flight distance constraint; if the projection overlap ratio is less than the preset overlap threshold, the target flight distance constraint corresponding to the current reliability quantification value remains unchanged.

[0296] The preset overlap threshold can be an empirical value. For example, let the current lateral distance threshold for the k-th pipe segment be... Compared to the current vertical distance threshold , The projection overlap ratio, The overlap threshold, To restore the marker, This refers to the range of the effective arc length. If... ,exist Under the premise of yes, perform recovery control, and Reset to and and will Set as ;like Maintain flight mode and , Unchanged, and maintained constant.

[0297] In this embodiment, the available conductor length is determined based on the reliability quantification value, the uniqueness judgment result, and the projection overlap ratio of two-dimensional and three-dimensional perception information. This can avoid interference from nearby conductors, tower materials, or weather scattering, and improve the reliability of the available conductor length. The control commands for the UAV's autonomous flight along the line are determined based on multiple constraints, including the flight mode under flight distance constraints, the available pipeline length, the cause of the anomaly, and the projection overlap ratio. This improves the safety and continuity of the UAV's autonomous flight along the line.

[0298] Figure 10 This is a partial inspection schematic diagram of an inspection pipeline provided in an embodiment of this application.

[0299] like Figure 10 As shown in the figure, a long black line represents the centerline of the pipeline corresponding to a portion of the detection pipeline, a small dashed box represents the two-dimensional sensing range of the pipeline corresponding to a portion of the detection pipeline, a large dashed box represents the three-dimensional sensing range of the pipeline corresponding to a portion of the detection pipeline, and a black triangle represents the inflection point marker of the pipeline in that portion of the detection pipeline.

[0300] The diagram includes five pipe segments corresponding to the partially inspected pipeline: segment 01, segment 02, segment 03, segment 04, and segment 05. Above each segment is the projection overlap ratio: 0.88 for segment 01, 0.68 for segment 02, 0.41 for segment 03, 0.92 for segment 04, and 0.66 for segment 05. Below each segment are the uniqueness determination result and anomaly type: 0.1 indicates "yes" for uniqueness, 0.2 indicates "yes" for uniqueness, 0.3 indicates "no" for uniqueness, and the anomaly type is "occlusion." The uniqueness determination results for segments 0.4 and 0.5 are both "yes."

[0301] The bottom part of the diagram shows the available wire lengths corresponding to some of the detection pipes, namely, segments 01 and 02 in the diagram.

[0302] Figure 11 This is a schematic diagram of the structure of a control device for a mobile device provided in an embodiment of this application, as shown below. Figure 11 As shown, the control device 1100 of the mobile device includes a processing module 1101, a communication module 1102, and a storage module 1103.

[0303] The processing module 1101 may be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor may include an application processor and a baseband processor. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0304] For example, processing module 1101 can be as follows: Figure 3 The processor 301 shown; the communication module 1102 can be as follows: Figure 3 The communication interface 303 shown; the storage module 1103 can be as follows: Figure 3 The internal memory 302 shown.

[0305] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0306] This application provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the mobile device control method described in the above method embodiments.

[0307] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the mobile device control method in the method flow shown in the above method embodiments.

[0308] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0309] Since the control device, computer-readable storage medium, and computer program product of the mobile device in the embodiments of the present invention can be applied to the above method, the technical effects obtained can also be referred to the above method embodiments. The embodiments of the present invention will not be repeated here.

[0310] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0311] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0312] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0313] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method for a mobile device, characterized in that, The method includes: Two-dimensional and three-dimensional perception information of a pipeline network under the same spatial reference is acquired through a mobile device; wherein, the pipeline network includes multiple detection pipelines, the two-dimensional perception information includes image sequences of multiple detection pipelines in the pipeline network, and the three-dimensional perception information includes the uniqueness determination result of the three-dimensional feature points corresponding to the detection pipeline for the detection pipeline; Based on the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point, the current reliability quantification value of the mobile device's perception capability is determined; wherein, different reliability quantification values ​​correspond to different flight distance constraints; the flight distance constraints are used to constrain the lateral and longitudinal flight distances of the mobile device relative to the detection pipeline; Based on the target flight distance constraint corresponding to the current reliability quantification value and the available pipeline length corresponding to the pipeline network, the target control command for the mobile device is determined.

2. The method according to claim 1, characterized in that, The determination of the current reliability quantification value of the mobile device's sensing capability based on the two-dimensional sensing information, the three-dimensional sensing information, and the uniqueness determination result of each of the three-dimensional feature points includes: Based on the two-dimensional perception information and the three-dimensional perception information, the projection overlap ratio is obtained; The projection overlap ratio, the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination results of each three-dimensional feature point are input into the perception reliability quantification model, and the current reliability quantification value of the perception capability of the mobile device is output.

3. The method according to claim 2, characterized in that, The three-dimensional sensing information includes the centerline of the detected pipeline and the three-dimensional sensing range; the two-dimensional sensing information includes the two-dimensional sensing range. The step of obtaining the projection overlap ratio based on the two-dimensional perception information and the three-dimensional perception information includes: The three-dimensional sensing information is fitted into a three-dimensional guide wire; Under the same spatial reference, based on the pipe centerline and the three-dimensional sensing range corresponding to the three-dimensional sensing information, the three-dimensional guide wire is projected onto the two-dimensional sensing range corresponding to the two-dimensional sensing information to obtain two-dimensional projection information; Based on the two-dimensional projection information and the two-dimensional sensing information, the overlapping conductor is determined, and the projection overlap ratio is obtained.

4. The method according to claim 2 or 3, characterized in that, The perceptual reliability quantification model includes a two-dimensional feature extraction network, a three-dimensional feature extraction network, and a fusion network. The step of inputting the projection overlap ratio, the two-dimensional perception information, the three-dimensional perception information, and the uniqueness determination result of each of the three-dimensional feature points into the perception reliability quantification model, and outputting the current reliability quantification value of the mobile device's perception capability, includes: The two-dimensional perception information is input into the two-dimensional feature extraction network to obtain the two-dimensional feature embedding vector corresponding to the two-dimensional perception information; The three-dimensional perception information is input into the three-dimensional feature extraction network to obtain the three-dimensional feature embedding vector corresponding to the three-dimensional perception information; The two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination result of each three-dimensional feature point are input into the fusion network to obtain the fusion feature vector; Based on the fused feature vector, the current reliability quantification value of the mobile device's sensing capability is determined.

5. The method according to claim 4, characterized in that, The step of inputting the two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination result of each three-dimensional feature point into the fusion network to obtain the fused feature vector includes: Calculate the difference vector between the two-dimensional feature embedding vector and the three-dimensional feature embedding vector; Calculate the product vector of the two-dimensional feature embedding vector and the three-dimensional feature embedding vector; The difference vector, the product vector, the two-dimensional feature embedding vector, the three-dimensional feature embedding vector, the projection overlap ratio, and the uniqueness determination results of each three-dimensional feature point are spliced ​​and fused to obtain the fused feature vector.

6. The method according to claim 4 or 5, characterized in that, The perceived reliability quantification model also includes a decision network; Determining the current reliability quantification value of the mobile device's sensing capability based on the fused feature vector includes: The fused feature vector is input into the decision network to obtain the current reliability quantization value of the mobile device's perception capability and the anomaly type corresponding to the current reliability quantization value.

7. The method according to any one of claims 1-6, characterized in that, The lower the current reliability quantification value of the mobile device's sensing capability, the greater the flight distance indicated by the flight distance constraint.

8. The method according to any one of claims 2-7, characterized in that, The method further includes: If the available pipeline length is greater than or equal to a preset length threshold, and the current reliability quantification value is within a first preset range, the target flight distance constraint condition corresponding to the current reliability quantification value remains unchanged. If the available pipeline length is less than the preset length threshold, and the current reliability quantification value is within the first preset range, the flight distance indicated in the target flight distance constraint corresponding to the current reliability quantification value is reduced to obtain the updated target flight distance constraint.

9. The method according to claim 8, characterized in that, The method further includes: Based on the anomaly type corresponding to the current reliability quantification value, determine the adjustment direction and adjustment range corresponding to the flight distance constraint; The flight distance constraint is adjusted based on the adjustment direction and adjustment range to obtain the updated target flight distance constraint.

10. The method according to claim 9, characterized in that, The anomaly types include two-dimensional perception information anomalies, three-dimensional perception information anomalies, and overlap anomalies; the adjustment directions include horizontal and vertical directions. The step of determining the adjustment direction and adjustment range corresponding to the flight distance constraint based on the anomaly type corresponding to the current reliability quantification value includes: If the anomaly type corresponding to the current reliability quantification value is the two-dimensional sensing information anomaly, the adjustment direction corresponding to the two-dimensional sensing information anomaly is determined to be vertical, and the adjustment range corresponding to the vertical is a preset first vertical range; If the anomaly type corresponding to the current reliability quantification value is the three-dimensional perception information anomaly, the adjustment direction corresponding to the three-dimensional perception information anomaly is determined to be horizontal, and the adjustment range corresponding to the horizontal is a preset second horizontal range; If the anomaly type corresponding to the current reliability quantification value is the overlapping anomaly, the adjustment direction corresponding to the overlapping anomaly is determined to be horizontal and vertical, the adjustment range corresponding to the horizontal is a preset third horizontal range, and the adjustment range corresponding to the vertical is a preset third vertical range.

11. The method according to any one of claims 2-10, characterized in that, The method further includes: If the projection overlap ratio is greater than or equal to the preset overlap threshold, the preset flight distance constraint condition will be used as the target flight distance constraint condition. If the projection overlap ratio is less than the preset overlap threshold, the target flight distance constraint corresponding to the current reliability quantification value remains unchanged.

12. The method according to any one of claims 1-11, characterized in that, The acquisition of two-dimensional and three-dimensional sensing information of the pipeline network under the same spatial reference via mobile devices includes: Initial two-dimensional sensing information is acquired through the two-dimensional sensing device of the mobile device; the two-dimensional sensing device includes an image acquisition device, and the initial two-dimensional sensing information includes an image sequence of the pipeline network; Initial three-dimensional perception information is acquired through the three-dimensional perception device of the mobile device; the three-dimensional perception device includes a lidar device, and the initial three-dimensional perception information includes point cloud data of the pipeline network; The initial two-dimensional sensing information and the initial three-dimensional sensing information are mapped to the same spatial reference to determine the pipe centerline, the two-dimensional sensing range corresponding to the initial two-dimensional sensing information, and the three-dimensional sensing range corresponding to the initial three-dimensional sensing information. Based on the centerline of the pipeline and the two-dimensional sensing range, the detected pipeline in the image sequence is divided into pipeline segments to obtain a first pipeline segment set; the first pipeline segment set includes multiple image sequence segments. Based on the centerline of the pipeline and the three-dimensional sensing range, the detected pipeline in the point cloud data is divided into pipeline segments to obtain a second pipeline segment set; the second pipeline segment set includes multiple point cloud data segments. Based on the first set of pipeline segments, the two-dimensional sensing information is obtained; The three-dimensional perception information is obtained based on the second set of pipeline segments.

13. The method according to claim 12, characterized in that, The two-dimensional sensing information includes directional consistency quantization value, longest continuous length, number of breaks, and inflection point markers; The step of obtaining the two-dimensional sensing information based on the first pipeline segment set includes: Based on the direction of the centerline of the pipeline, obtain the directional consistency quantization value corresponding to each image sequence segment in the first pipeline segment set; Based on the wire pixels of the image sequence segments along the centerline direction of the pipeline, the longest continuous length and the number of breaks corresponding to each image sequence segment in the first pipeline segment set are obtained; Based on preset inflection point determination conditions, the inflection point markers corresponding to each image sequence segment in the first pipeline segment set are determined; Based on the directional consistency quantization value, the longest continuous length, the number of breaks, and the inflection point marker, the two-dimensional sensing information corresponding to the first pipeline segment set is determined.

14. The method according to claim 13, characterized in that, The step of determining the inflection point markers corresponding to each image sequence segment in the first pipeline segment set based on preset inflection point determination conditions includes: If the direction change rate of the image sequence segment is greater than or equal to a preset direction change rate threshold, the occlusion segment of each image sequence segment in the first pipeline segment set is determined; Based on the occlusion section, the occlusion arc length of each image sequence segment in the first pipeline segment set is obtained; Based on the occlusion arc length, the inflection point markers corresponding to each image sequence segment in the first pipeline segment set are determined.

15. The method according to claim 12, characterized in that, The three-dimensional sensing information includes a first distribution uniformity quantization value, an echo stability quantization value, and a first fine-tuning stability quantization value. The acquisition of the three-dimensional perception information based on the second pipeline segment set includes: Based on the direction of the pipeline centerline, the number of three-dimensional feature points of each point cloud data segment in the second pipeline segment set within a preset length is determined, and the first distribution uniformity quantization value of each point cloud data segment in the second pipeline segment set is obtained. Based on the echo intensity of each point cloud data segment in the second pipeline segment set, determine the echo stability quantization value of each point cloud data segment in the second pipeline segment set; Based on the preset fine-tuning amplitude and the three-dimensional perception range of the point cloud data segment, the first fine-tuning stability quantization value of each point cloud data segment in the second pipeline segment set is obtained; Based on the first distribution uniformity quantization value, the echo stability quantization value, and the first fine-tuning stability quantization value, the three-dimensional sensing information corresponding to the second pipeline segment set is determined.

16. The method according to any one of claims 1-15, characterized in that, The method further includes: Based on a preset offset pipeline centerline parallel to the pipeline centerline, the second distribution uniformity quantization value and the second fine-tuning stability quantization value of each point cloud data segment in the offset second pipeline segment set are obtained. Based on the first distribution uniformity quantization value, the first fine-tuning stability quantization value, the second distribution uniformity quantization value, and the second fine-tuning stability quantization value, the uniqueness determination result of each point cloud data segment in the second pipeline segment set is obtained by using a preset uniqueness quantization formula.

17. The method according to claim 16, characterized in that, The method further includes: If the uniqueness quantization value is greater than or equal to the uniqueness quantization threshold, the uniqueness determination result of each point cloud data segment in the second pipeline segment set is determined to be yes; the uniqueness determination result being yes indicates that the reliability of each point cloud data segment in the second pipeline segment set is high. If the uniqueness quantization value is less than the uniqueness quantization threshold, the uniqueness determination result of each point cloud data segment in the second pipeline segment set is determined to be no; the uniqueness determination result being no indicates that the reliability of each point cloud data segment in the second pipeline segment set is low.

18. The method according to any one of claims 1-17, characterized in that, The method further includes: If the current reliability quantification value is within the second preset range, the projection overlap ratio is greater than or equal to the preset overlap threshold, and the uniqueness determination result is yes, then the available pipeline length corresponding to the pipeline network is determined.

19. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the control method of the mobile device as described in any one of claims 1 to 18.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the control method of the mobile device as described in any one of claims 1 to 18.

21. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by a processor, implement the control method for a mobile device as described in any one of claims 1 to 18.

Images