An obstacle reconstruction method, device, terminal equipment and storage medium

Abstract

Embodiments of the present application disclose an obstacle reconstruction method and device, a terminal device and a storage medium. After a user selects a target image and a target obstacle type on the terminal device, the user can operate the target image to send an operation instruction to the terminal device, so that the terminal device reconstructs the target obstacle according to the operation instruction, a projection matrix of the target image and a target obstacle reconstruction rule. The obstacle reconstruction method and device can support different obstacle types and have different reconstruction rules for different obstacle types. The terminal device can accurately reconstruct the obstacle by interacting with the user and calling the corresponding reconstruction rule, thereby improving the obstacle reconstruction effect and solving the problem of poor obstacle reconstruction effect in the prior art.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of three-dimensional reconstruction, and in particular to an obstacle reconstruction method and device, a terminal device, and a storage medium. BACKGROUND

[0002] Currently, in order to ensure the safety of an unmanned device during autonomous operation, a user first needs to survey a to-be-operated plot, and the surveying content includes the boundary of the to-be-operated plot and the position and height of obstacles in the to-be-operated plot. Then the user needs to construct a map according to the surveying result and import the map into the unmanned device for path planning and navigation obstacle avoidance.

[0003] In the prior art, the reconstruction effect of obstacles in the constructed map is poor, some obstacles cannot be completely reconstructed, and some obstacles cannot be reconstructed at all, which leads to a major safety hazard of the unmanned device during autonomous operation. How to improve the reconstruction effect of obstacles has become a technical problem to be solved at present. SUMMARY

[0004] Embodiments of the present application provide an obstacle reconstruction method, device, terminal device, and storage medium, which solve the technical problem of poor obstacle reconstruction effect in the prior art and can accurately reconstruct obstacles.

[0005] In a first aspect, embodiments of the present application provide an obstacle reconstruction method, comprising:

[0006] receiving at least two images, and calculating a projection matrix of each image, wherein each image includes a target obstacle to be reconstructed;

[0007] in response to an image selection instruction, determining a preset number of target images in the images;

[0008] in response to an obstacle type selection instruction, determining a target obstacle type in the obstacle types, and calling a corresponding target obstacle reconstruction rule according to the target obstacle type;

[0009] in response to an operation instruction on the target image, reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule.

[0010] In a second aspect, embodiments of the present application provide an obstacle reconstruction device, comprising:

[0011] a projection matrix calculation module configured to receive at least two images and calculate a projection matrix of each image, wherein each image includes a target obstacle to be reconstructed;

[0012] an image selection module configured to determine a preset number of target images in the images in response to an image selection instruction;

[0013] a type selection module, configured to determine a target obstacle type from the obstacle types in response to an obstacle type selection instruction, and invoke a corresponding target obstacle reconstruction rule according to the target obstacle type;

[0014] a reconstruction module, configured to reconstruct the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule in response to an operation instruction on the target image.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, which comprises a processor and a memory;

[0016] The memory is configured to store a computer program and transmit the computer program to the processor;

[0017] The processor is configured to execute the obstacle reconstruction method according to the instructions in the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the obstacle reconstruction method according to the first aspect.

[0019] According to the above, the embodiment of the present application provides an obstacle reconstruction method. After a user selects a target image and a target obstacle type on a terminal device, the user can send an operation instruction to the terminal device by operating the target image, so that the terminal device reconstructs the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule. The way of reconstructing the obstacle by the interaction between the user and the terminal device can support different obstacle types, and has different reconstruction rules for different obstacle types. The terminal device can accurately reconstruct the obstacle by the interaction with the user and the corresponding reconstruction rule, thereby improving the reconstruction effect of the obstacle. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of an obstacle reconstruction method provided by the embodiment of the present application.

[0021] Figure 2 A schematic diagram of a user selecting a target image provided by the embodiment of the present application.

[0022] Figure 3 A schematic diagram of a user selecting a target obstacle type provided by the embodiment of the present application.

[0023] Figure 4 A schematic diagram of a user operating a target image provided by the embodiment of the present application.

[0024] Figure 5 A schematic diagram for a user to label the bottom position of a utility pole is provided for an embodiment of the present application.

[0025] Figure 6 A schematic diagram for a user to input the radius of a utility pole is provided for an embodiment of the present application.

[0026] Figure 7 A schematic diagram for constructing a rectangular frame is provided for an embodiment of the present application.

[0027] Figure 8 A schematic diagram for setting the radius of a tree is provided for an embodiment of the present application.

[0028] Figure 9 A schematic diagram for the range of a first circle is provided for an embodiment of the present application.

[0029] Figure 10 A schematic diagram for changing the shape of a third line segment is provided for an embodiment of the present application.

[0030] Figure 11 A schematic diagram for dragging a split point of a third line segment is provided for an embodiment of the present application.

[0031] Figure 12 A schematic diagram for dragging a midpoint of a third line segment is provided for an embodiment of the present application.

[0032] Figure 13 A flowchart of another obstacle reconstruction method is provided for an embodiment of the present application.

[0033] Figure 14 A schematic diagram for correcting the position of a target obstacle is provided for an embodiment of the present application.

[0034] Figure 15 A schematic diagram of a vectorized map is provided for an embodiment of the present application.

[0035] Figure 16 A structural schematic diagram of an obstacle reconstruction device is provided for an embodiment of the present application.

[0036] Figure 17 A structural schematic diagram of a terminal device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following description and drawings are illustrative of specific embodiments of the application and are not intended to be limiting thereof. The embodiments are presented by way of example only. Separate components and functions are optional and the order of operations can vary. Parts and features of some embodiments can be included or substituted in other embodiments. The scope of the embodiments of the application encompasses the entire scope of the claims and all available equivalents of the claims. In this document, the terms "cha racterized by," "determined by," "comprised of," and the like, are intended to encompass the items listed in their entirety, as well as equivalent items not explicitly listed. In this document, the terms "first," "second," "third," etc. are used merely as identifiers for distinguished items, and are not intended to be taken literally to denote a physical order or position. In this document, the term "comprising" is intended to mean "including, but not limited to," and is used in the sense m which the phrase "consists of plus the listed items is intended to cover the listed items as well as equivalents thereof. In this document, various examples are described using a progression of

[0038] At present, there are generally three methods for reconstructing obstacles in the existing art: 1. using a three-dimensional reconstruction method, that is, collecting images and RTK data of the to-be-operated plot by an unmanned device, reconstructing the poses of all images and a sparse point cloud of the three-dimensional scene using SFM (Structure From Motion), and usually the sparse point cloud cannot clearly obtain the obstacles, so the point cloud needs to be densified using the MVS (Multi-views Stereo) technology to realize the reconstruction of the obstacles, and then the position of the obstacles is located by an AI / artificial recognition method. 2. using a handheld surveying instrument to mark points, the handheld surveying instrument includes an RTK (Real-time kinematic) module, which can record the absolute geographic coordinates of the marked points, and the user marks points on the bottom of the obstacle to determine the real coordinate position, and then estimates the height of the obstacle to realize the reconstruction of the obstacle. 3. using additional sensors such as a laser radar or an RGB-D camera to obtain depth information of the obstacle, and reconstructing the obstacle by SLAM (Simultaneous Localization and Mapping) / SFM technology.

[0039] However, the obstacle reconstruction methods in the existing art all have obvious defects. Among them, the method of reconstructing obstacles based on images and RTK data collected by an unmanned device usually cannot realize complete recovery of columnar obstacles such as power poles, and cannot recover the position and sag of the power lines, resulting in a crashed unmanned device when operating, time delay and high after-sales cost. The method of reconstructing obstacles based on a handheld surveying instrument usually can only measure the bottom of the obstacle, cannot measure the height of the obstacle, and the user can only guess the height of the obstacle according to experience, and when the guess is not accurate, the unmanned device is easy to crash into the obstacle, and in order to ensure safety, the height of the obstacle is set to be very high, which affects the operation efficiency. The method of reconstructing obstacles using a laser radar or an RGB-D camera can effectively recover obstacles such as trees and power poles with large volume, but it is difficult to identify obstacles such as power lines and diagonal lines with small volume, so it cannot effectively reconstruct power lines and diagonal lines, resulting in a crashed unmanned device when operating.

[0040] In summary, the obstacle reconstruction methods in the existing art cannot completely reconstruct some obstacles, and cannot reconstruct some obstacles at all, resulting in a major safety hazard in the autonomous operation of the unmanned device. In order to solve the above technical problems, the embodiments of the present application provide an obstacle reconstruction method as shown in Figure 1 Figure 1 ​A flowchart of an obstacle reconstruction method provided by an embodiment of the present application is shown. The obstacle reconstruction method provided by the embodiment of the present application can be executed by a terminal device, which can be implemented by software and / or hardware. The terminal device can be composed of two or more physical entities, or can be composed of one physical entity, such as a computer, a mobile phone, a tablet, and the like. The method includes the following steps:

[0041] In step 101, at least two images are received, and a projection matrix of each image is calculated, wherein each image includes a target obstacle to be reconstructed.

[0042] In the obstacle reconstruction, the user first needs to input at least two images to the terminal device, and each of the input images includes an overlapping part, and the target obstacle to be reconstructed can be observed on each image. For example, when the target obstacle is a telegraph pole, the telegraph pole corresponding picture is included in each image.

[0043] After receiving the images, the terminal device further calculates the projection matrix of each image, so as to project the pixel points in each image into a three-dimensional space by using the projection matrix. Specifically, each image input by the user in the embodiment is derived from the same camera, and the pose data of the camera when shooting the image is included in each image, wherein the pose data of the camera is the specific position of the camera in the world coordinate system, including the rotation and translation of the xyz coordinate axis. The terminal device can calculate the projection matrix of each image according to the pose data corresponding to each image. For example, two input images are denoted as I1 and I2, the pose data corresponding to the two images is denoted as T1 and T2, the intrinsic matrix of the camera is denoted as K, and the projection matrices P1 and P2 corresponding to the two images I1 and I2 are respectively:

[0044]

[0045] wherein (1:3,∶) represents taking the first row to the third row and taking all columns, represents the jth row vector of the ith projection matrix.

[0046] In step 102, in response to an image selection instruction, a preset number of target images are determined in the images.

[0047] After the user inputs the images to the terminal device, the user can select a preset number of images from the input images by sending an image selection instruction to the terminal device. In an embodiment, the terminal device includes a display screen, and the display screen displays a graphical user interface. The graphical user interface is an interface display format for communication between a user and an electronic device. The graphical user interface is composed of windows, drop-down menus, dialog boxes, and corresponding control mechanisms. In the graphical user interface, the objects that the user sees and operates are graphical objects. The user can interact with the terminal device by operating the graphical objects in the graphical user interface, thereby issuing various instructions to the terminal device. For example, the user can use a mouse or other input device to manipulate the graphical objects displayed on the graphical user interface to select commands, call files, start programs, or perform other daily tasks. When the display screen is a touch screen, the user can also operate the graphical objects on the touch screen by gestures. For example, the user can click an image on the graphical user interface by a mouse or a gesture to select the image. Figure 2 Figure 2 As shown in the schematic diagram of the graphical user interface provided in the embodiment of the present application, the graphical user interface displays the images input by the user. The user can send an image selection instruction to the terminal device by clicking an image on the graphical user interface by a mouse or a gesture. After receiving the image selection instruction, the terminal device can determine the target image selected by the user and enclose the target image with a box. In this embodiment, the preset number of target images selected by the user is two, so that the real position of the target obstacle in the three-dimensional space can be calculated subsequently, and the user is prevented from performing too many operations on the target images.

[0048] In step 103, in response to the obstacle type selection instruction, the target obstacle type is determined from the obstacle types, and the corresponding target obstacle reconstruction rule is called according to the target obstacle type.

[0049] After the user selects the target images, the user also needs to further select the target obstacle type corresponding to the target obstacle, so that the terminal device can call the corresponding obstacle reconstruction rule for reconstruction subsequently. Similarly, in this embodiment, the user can send an obstacle type selection instruction to the terminal device by interacting with the graphical user interface to select the target obstacle type corresponding to the target obstacle. In an embodiment, the obstacle type includes at least one of a power pole, a diagonal line, a tree, and a power line. The diagonal line refers to a line segment arranged obliquely, such as a steel wire used to fix a power pole. One end of the steel wire is fixed to the top of the power pole, and the other end is fixed to the ground. For example, as shown in Figure 3 ​As shown, after the user selects the target image, the right side of the graphical user interface displays a barrier type selection box, and the barrier type selection box displays a plurality of barrier types for the user to select. After the terminal device determines the barrier type selected by the user, it can take the barrier type as the target barrier type corresponding to the target barrier, and then call the corresponding target barrier reconstruction rule according to the target barrier type, so as to subsequently reconstruct the target barrier according to the target barrier reconstruction rule.

[0050] Step 104, in response to the operation instruction on the target image, reconstructing the target barrier according to the operation instruction, the projection matrix of the target image, and the target barrier reconstruction rule.

[0051] After the user selects the target barrier type, the user can further reconstruct the target barrier on the terminal device. Specifically, after the terminal device calls the target barrier reconstruction rule, it displays the preset number of target images selected by the user on the graphical user interface. During the process of reconstructing the target barrier, for different target barrier types, the user can use different interactive ways to operate the preset number of target images on the graphical user interface, thereby sending operation instructions to the terminal device. After the terminal device receives the operation instructions, it can execute the corresponding steps in the target barrier reconstruction rule according to the user's operation, and calculate the real position of the target barrier in the three-dimensional space according to the projection matrix of the target image, thereby completing the reconstruction of the target barrier.

[0052] For example, the target barrier to be reconstructed is a telegraph pole, and the terminal device displays two target images including the telegraph pole on the graphical user interface, as shown in Figure 4 The user can operate the telegraph pole in the two target images, for example, mark the top and bottom of the telegraph pole in the two target images, and the terminal device can calculate the real position of the top of the telegraph pole in the three-dimensional space and the real position of the bottom of the telegraph pole in the three-dimensional space according to the target barrier reconstruction rule corresponding to the telegraph pole and the projection matrix of the target image, thereby determining the position and length of the telegraph pole. Then the user can input the radius of the telegraph pole on the graphical user interface, and the terminal device can reconstruct the telegraph pole in the three-dimensional space according to the position, length and radius of the telegraph pole. Finally, the user can export the reconstructed telegraph pole as a vector barrier and add the telegraph pole to the vectorized map.

[0053] The above, the embodiment of the application provides a kind of obstacle reconstruction method, user selects target image and target obstacle type on terminal device, can be operated to target image to send operation instruction to terminal device, so that terminal device is according to operation instruction, the projection matrix of target image and target obstacle reconstruction rule, the reconstruction of target obstacle is carried out.The way of the embodiment of the application is realized obstacle reconstruction by user and terminal device interaction, can support different obstacle types, and it has different reconstruction rules for different obstacle types, and terminal device can accurately reconstruct obstacle by interaction with user and calling corresponding reconstruction rule, improve the reconstruction effect of obstacle.

[0054] The embodiment of the application also provides another obstacle reconstruction method, the obstacle reconstruction method provided in the embodiment is the embodiment of the above obstacle reconstruction method, comprising:

[0055] Step 201, receive at least two images, calculate the projection matrix of each image, wherein each image includes target obstacle to be reconstructed.

[0056] Step 202, in response to image selection instruction, determine a preset number of target images in the image.

[0057] Step 203, in response to obstacle type selection instruction, determine target obstacle type in obstacle type, and call corresponding target obstacle reconstruction rule according to target obstacle type.

[0058] Step 204, in response to operation instruction on target image, reconstruct target obstacle according to operation instruction, projection matrix of target image and target obstacle reconstruction rule.

[0059] In one embodiment, when the target obstacle type is a power pole, in response to the operation instruction on the target image in step 204, reconstruct the target obstacle according to the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule, comprising:

[0060] Step 2041, in response to power pole bottom annotation instruction, determine the bottom position of the power pole in the target image, and calculate the first spatial coordinates of the bottom of the power pole according to the projection matrix of the target image.

[0061] In one embodiment, when the target obstacle type is a power pole, the user first needs to mark the bottom position of the power pole in the target image displayed on the graphical user interface, and send the power pole bottom annotation instruction to the terminal device in the graphical user interface, so that the terminal device can confirm the bottom position of the power pole in the target image. For example, Figure 5As shown, two target images I1 and I2 are displayed on the graphical user interface, and the user can mark the bottom position of the power pole in I1 and I2 by operating the mouse. After confirming the marking, the bottom position of the power pole is displayed as a circle, and the terminal device receives the marking instruction of the bottom position of the power pole from the user and determines the bottom position of the power pole marked by the user in I1 and I2.

[0062] Then, the terminal device can further calculate the first spatial coordinates of the bottom of the power pole in the world coordinate system. Specifically, the terminal device can calculate the first spatial coordinates of the bottom of the power pole in the world coordinate system according to the bottom position of the power pole in I1 and I2 and the projection matrix corresponding to I1 and I2, respectively.

[0063] In one embodiment, the calculation of the first spatial coordinates of the bottom of the power pole according to the projection matrix of the target image in step 2041 comprises:

[0064] In step 20411, the first coordinates of the pixels occupied by the bottom position of the power pole in the target image are determined, and the first spatial coordinates of the bottom of the power pole are calculated according to the first coordinates and the projection matrix of the target image.

[0065] When calculating the first spatial coordinates of the bottom of the power pole, the terminal device first needs to determine the first coordinates of the pixels occupied by the bottom position of the power pole in the two target images I1 and I2, respectively. Specifically, the terminal device can establish a two-dimensional coordinate system in I1 and I2, respectively, and determine the first coordinates according to the position of the pixels occupied by the bottom position of the power pole. Assuming that the first coordinates corresponding to I1 and I2 are x1=(u1, v1) and x2=(u2, v2), respectively. T T where x i is the first coordinates corresponding to the i-th target image, and u i and v i represent the x-axis coordinate and y-axis coordinate of the first coordinates of the i-th target image, respectively.

[0066] Then, the terminal device can calculate the first spatial coordinates of the bottom of the power pole in the world coordinate system according to x1, x2, and the projection matrix corresponding to I1 and I2. Specifically, assuming that the first spatial coordinates are M=(m x , m y , m z ) T , the optimization equation can be obtained based on the triangulation principle:

[0067]

[0068] SVD decomposition of the above constraint equation AM=0 can be performed to obtain the first spatial coordinates of the bottom of the power pole as M. ​

[0069] Step 2042, in response to the top of the pole marking instruction, determining the top position of the pole in the target image, calculating the second spatial coordinates of the top of the pole according to the projection matrix of the target image.

[0070] Similarly, for the second spatial coordinates of the top of the pole, the user can mark the top position of the pole in I1 and I2, and the terminal device can calculate the second spatial coordinates N of the top of the pole according to the second coordinates of the pixels occupied by the top position of the pole in I1 and I2 and the projection matrix of I1 and I2. The specific process is similar to the process of calculating the first spatial coordinates M, which will not be described in detail in this embodiment.

[0071] Step 2043, in response to the first radius input instruction, determining the radius of the pole.

[0072] Since the pole is not a line segment but a cylinder with a certain thickness, and the thickness of the pole at different positions is not consistent, the user can input the radius of the pole. For example, the user can input the radius of the pole on the right side of the graphical user interface shown in the figure, and click the "OK" button after inputting to send the first radius input instruction to the terminal device. The terminal device can determine the radius of the pole according to the data input by the user. Figure 6

[0073] Step 2044, completing the reconstruction of the pole according to the first spatial coordinates, the second spatial coordinates and the radius of the pole.

[0074] Finally, the terminal device can complete the reconstruction of the pole according to the first spatial coordinates M, the second spatial coordinates N and the radius of the pole. Specifically, the terminal device can determine the position and length of the pole according to the first spatial coordinates M and the second spatial coordinates N, and generate a cylinder according to the radius of the pole to complete the reconstruction of the pole.

[0075] On the basis of the above embodiment, the step 2044 of completing the reconstruction of the pole according to the first spatial coordinates, the second spatial coordinates and the radius of the pole includes:

[0076] Step 20441, constructing a first line segment according to the first spatial coordinates and the second spatial coordinates.

[0077] First, the terminal device connects the first spatial coordinates M and the second spatial coordinates N to construct a first line segment MN.

[0078] Step 20442, taking the first line segment as the central axis of the pole, generating the pole according to the radius of the pole.

[0079] ​The terminal device takes the first line segment MN as the central axis of the electric pole, and generates a cylinder containing the head and tail and the radius according to the radius of the electric pole, so as to complete the reconstruction of the electric pole. Then the terminal device can further export the electric pole as a vector obstacle and write it into the required output document, so that the user can obtain the data of the reconstructed electric pole according to the document output by the terminal device.

[0080] In the above, the embodiment of the present application provides an obstacle reconstruction method. After the user selects a target image and selects the target obstacle type as an electric pole on the terminal device, the user can label the bottom position and the top position of the electric pole by operating the target image. Then the terminal device can calculate the first spatial coordinates of the bottom and the second spatial coordinates of the top of the electric pole according to the projection matrix of the target image. Finally, the reconstruction of the electric pole is completed according to the first spatial coordinates, the second spatial coordinates and the radius of the electric pole input by the user. The way of realizing the reconstruction of the obstacle by the interaction between the user and the terminal device can accurately reconstruct the electric pole and improve the reconstruction effect of the electric pole.

[0081] In one embodiment, when the target obstacle type is a diagonal line, the reconstruction of the target obstacle is performed according to the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule in response to the operation instruction on the target image in step 204, including:

[0082] Step 2045, determining the bottom position of the diagonal line in the target image in response to the diagonal line bottom labeling instruction, and calculating the third spatial coordinates of the bottom of the diagonal line according to the projection matrix of the target image.

[0083] In one embodiment, when the target obstacle type is a diagonal line, the reconstruction way of the diagonal line is essentially consistent with that of the electric pole. The user first labels the bottom position of the diagonal line in the target image displayed on the graphical user interface, and sends a diagonal line bottom labeling instruction to the terminal device in the graphical user interface, so that the terminal device can confirm the bottom position of the diagonal line in the target image. Then, the terminal device can further determine the third coordinates according to the position of the pixels occupied by the bottom position of the diagonal line in the target image, and calculate the third spatial coordinates of the bottom of the diagonal line in the world coordinate system according to the third coordinates and the conversion matrix of the target image. The process can refer to the process of calculating the first spatial coordinates of the electric pole, which will not be described in detail in this embodiment.

[0084] Step 2046, determining the top position of the diagonal line in the target image in response to the diagonal line top labeling instruction, and calculating the fourth spatial coordinates of the top of the diagonal line according to the projection matrix of the target image.

[0085] Step 2047, constructing a second line segment according to the third spatial coordinates and the fourth spatial coordinates to complete the reconstruction of the diagonal line.

[0086] After the terminal device calculates the third spatial coordinates of the bottom of the diagonal line and the fourth spatial coordinates of the bottom, since the diagonal line is not as thick as a telegraph pole, it is not necessary to set the radius, and after connecting the third spatial coordinates and the fourth spatial coordinates to construct a second line segment, the reconstruction of the diagonal line is completed.

[0087] On the basis of the above-mentioned embodiments, when the number of diagonal lines is more than one, further comprising:

[0088] Step 2048, determining a first obstacle range according to the third spatial coordinates of each diagonal line and the fourth spatial coordinates of each diagonal line.

[0089] When the number of reconstructed diagonal lines is more than one, since the unmanned device cannot pass through the middle of the adjacent two diagonal lines, it is not necessary to carefully derive all the diagonal lines. In this embodiment, the first obstacle range can be further determined according to the third spatial coordinates of each diagonal line and the fourth spatial coordinates of each diagonal line. For example, as shown in FIG. 8, the cylinder represents a telegraph pole, and the line represents two diagonal lines. After the third spatial coordinates and the fourth spatial coordinates of the two diagonal lines are determined respectively, a rectangular frame that can enclose the two diagonal lines can be determined according to the third spatial coordinates and the fourth spatial coordinates of the two diagonal lines, and the first obstacle range can be determined according to the boundary points of the rectangular frame in the world coordinate system. When the number of diagonal lines is three or more, a planar rectangular frame cannot include three diagonal lines. At this time, a cuboid can be generated to include multiple diagonal lines in the cuboid, and the first obstacle range can be determined according to the boundary of the cuboid. In addition, when multiple diagonal lines are not gathered in one position, multiple rectangular frames or cuboids can be generated. At this time, the first obstacle range has multiple. Figure 7

[0090] Step 2049, generating an obstacle according to the first obstacle range.

[0091] Finally, the first obstacle range can be used as a prohibited passage area, and the rectangular frame or cuboid corresponding to the first obstacle range can be derived as an obstacle, so that the subsequent unmanned device can use the first obstacle range as a prohibited passage area when performing work, thereby avoiding the unmanned device from colliding with the diagonal line and causing the unmanned device to explode.

[0092] ​According to the above, the embodiment of the present application provides an obstacle reconstruction method, after a user selects a target image and selects a target obstacle type as a diagonal line on a terminal device, the user can label a bottom position and a top position of the diagonal line by operating the target image, then the terminal device can calculate a first space coordinate of the bottom of the diagonal line and a second space coordinate of the top of the diagonal line according to a projection matrix of the target image, and finally reconstruct the diagonal line according to the first space coordinate and the second space coordinate. In addition, when there are multiple diagonal lines, the terminal device can further generate a rectangular frame or a cuboid including the multiple diagonal lines, determine a first obstacle range according to an area included by the rectangular frame or the cuboid, and generate an obstacle according to the first obstacle range. The embodiment of the present application can accurately reconstruct the diagonal line by the interaction between the user and the terminal device, and improves the reconstruction effect of the diagonal line.

[0093] On the basis of the above embodiment, when the target obstacle type is a tree, in step 204, in response to the operation instruction on the target image, the target obstacle is reconstructed according to the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule, including:

[0094] In step 20410, in response to a tree bottom labeling instruction, a bottom position of the tree is determined in the target image, and a fifth space coordinate of the bottom of the tree is calculated according to the projection matrix of the target image.

[0095] In an embodiment, when the target obstacle type is a tree, the reconstruction manner of the tree is essentially consistent with that of the power pole. The user first needs to label a bottom position of the tree in the target image displayed on the graphical user interface, and sends a tree bottom labeling instruction to the terminal device in the graphical user interface, so that the terminal device can confirm the bottom position of the tree in the target image. Then, the terminal device can further determine a fifth coordinate according to the position of the pixel occupied by the bottom position of the tree in the target image, and calculate a fifth space coordinate of the bottom of the tree in the world coordinate system according to the fifth coordinate and the conversion matrix of the target image. The process can refer to the process of calculating the first space coordinate of the power pole, which will not be described herein again.

[0096] In step 20411, in response to a tree top labeling instruction, a top position of the tree is determined in the target image, and a sixth space coordinate of the top of the tree is calculated according to the projection matrix of the target image.

[0097] In step 20412, in response to a second radius input instruction, a radius of the tree is determined.

[0098] Similarly, since the tree is not a line segment and the thicknesses of different trees are inconsistent, the user can input the radius of the tree by himself / herself, and the terminal device can determine the radius of the tree according to the data input by the user.

[0099] On the basis of the above embodiment, in response to the second radius input instruction, determining the radius of the tree in step 20412 includes:

[0100] In step 204121, in response to the second radius input instruction, a first circle is generated at the bottom position of the tree in the specified target image.

[0101] In one embodiment, the difference between the reconstruction of the tree and the reconstruction of the pole by the user is that the reconstruction of the tree supports a more flexible radius setting method. As shown in Figure 8 , the user can drag a circle in the target image at the bottom position of the tree in the target image displayed on the graphical user interface by mouse or gesture operation, and the circle has the same center as the trunk of the tree. After the drag operation is completed, the terminal device can receive the second radius input instruction sent by the user, and the terminal device generates a first circle at the bottom position of the tree operated by the user. In addition, it should be noted that the range of the first circle in the target image is not equal to the range of the trunk, but needs to be greater than all the ranges covered by the tree in the top view direction, as shown in Figure 9 .

[0102] In step 204122, the radius of the tree is determined according to the radius of the first circle.

[0103] After the first circle is generated on the target image operated by the user, the terminal device can calculate the real radius of the first circle in the world coordinate system according to the first circle on the target image, and take the radius as the radius of the tree. Specifically, the radius of the tree is determined according to the radius of the first circle, including:

[0104] In step 2041221, the height of the tree is determined according to the fifth spatial coordinate and the sixth spatial coordinate.

[0105] First, the terminal device determines the height h of the tree according to the z-axis coordinate in the fifth spatial coordinate and the z-axis coordinate in the sixth spatial coordinate.

[0106] In step 2041222, the first pixel number occupied by the tree in the height direction is determined in the specified target image, and a first ratio between the height and the first pixel number is determined.

[0107] Then, the first pixel number s occupied by the tree in the height direction is determined in the target image operated by the user. It should be noted that when determining the first pixel number s, only the first pixel number of a line segment of the tree in the height direction needs to be determined, and the first pixel number of all regions of the tree in the height direction does not need to be determined. Then, a first ratio t between the height h of the tree and the first pixel number s can be calculated, t = h / s.

[0108] In step 2041, the second pixel number occupied by the radius of the first circle in the specified image is determined, and the second pixel number is multiplied by the first ratio to obtain the radius of the tree.

[0109] After the first ratio is determined, the terminal device further determines the second pixel number k occupied by the radius of the first circle in the target image operated by the user, and then multiplies the second pixel number k by the first ratio t to obtain the real radius of the tree in the world coordinate system.

[0110] In step 2041, the second pixel number occupied by the radius of the first circle in the specified image is determined, and the second pixel number is multiplied by the first ratio to obtain the radius of the tree.

[0111] Finally, the terminal device connects the fifth spatial coordinate and the sixth spatial coordinate to generate a line segment, takes the line segment as a central axis, and generates a cylindrical body containing a head, a tail and a radius according to the radius of the tree, thereby completing the reconstruction of the tree.

[0112] The above, the embodiment of the present application provides a kind of obstacle reconstruction method, after user selects target image and selects target obstacle type as tree on terminal device, the bottom position and top position of tree can be marked by operating target image, then the first spatial coordinate of the bottom of tree and the second spatial coordinate of top can be calculated according to the projection matrix of target image by terminal device, finally, the reconstruction of tree is completed according to the first spatial coordinate, the second spatial coordinate and the radius of tree input by user.The way of the present application embodiment for realizing obstacle reconstruction by user and terminal device interaction, tree can be accurately reconstructed, and the reconstruction effect of tree is improved.In addition, in the present application embodiment, when the radius of tree is input, radius is set in the way of drag circle, so that user can set when the radius of tree is set, reference is made to the range covered by tree in target image, and the accuracy of the set radius is improved.

[0113] In one embodiment, when the target obstacle type is a wire, the reconstruction of the target obstacle is performed according to the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule in response to the operation instruction of the target image in step 204, including:

[0114] Step 20414, in response to the top of the pole marking instruction, determining the top positions of the two poles connected by the electric wire in the target image, and generating a third line segment connecting the top positions of the two poles.

[0115] In one embodiment, when the target obstacle type is electric wire, since the electric wire is generally connected between two poles, the user can mark the top positions of the poles in the two target images displayed on the graphical user interface. The marking process can refer to the above-mentioned embodiment of marking the top of the pole, which will not be described herein again. Then, the terminal device can generate a third line segment connecting the top positions of the two poles in the two target images according to the top positions of the poles marked by the user.

[0116] Step 20415, in response to the line segment dragging instruction, changing the shape of the third line segment in the target image, and completing the reconstruction of the electric wire according to the third line segment after the shape is changed, the top positions of the two poles, and the projection matrix of the target image.

[0117] Then, in order to make the shape and position of the third line segment in the target image as close as possible to the electric wire displayed in the target image, the user needs to operate the third line segment in the two target images to send a line segment dragging instruction to the terminal device, and the terminal device changes the shape of the third line segment according to the line segment dragging instruction. For example, the user can drag the third line segment in the two target images displayed on the graphical user interface by using a mouse to send a line segment dragging instruction to the terminal device, and the terminal device changes the shape of the third line segment according to the dragging operation of the user. After the user finishes the dragging, the third line segment after the shape is changed can be obtained in the target image, as shown in FIG. 6C. Figure 10 Subsequently, the terminal device can calculate the coordinates of the top of the pole in the three-dimensional space according to the top positions of the two poles in the target image and the projection matrix of the target image, and calculate the coordinates of the third line segment in the three-dimensional space according to the position of the third line segment in the target image after the shape is changed and the projection matrix of the target image. After connecting the coordinates of the third line segment and the coordinates of the top of the pole, the reconstruction of the electric wire can be completed.

[0118] Based on the above-mentioned embodiments, in step 20415, in response to the line segment dragging instruction, the shape of the third line segment is changed in the target image, and the reconstruction of the electric wire is completed according to the third line segment after the shape is changed, the top positions of the two poles, and the projection matrix of the target image, including:

[0119] Step 204151, determining a plurality of segmentation points on the third line segment in the target image, and the plurality of segmentation points are used to divide the third line segment into a plurality of sub-line segments.

[0120] In one embodiment, after the terminal device generates the third line segment on the target image, the terminal device first determines a plurality of segmentation points in the third line segment on each target image, wherein the plurality of segmentation points are used to divide the third line segment into a plurality of sub-line segments, and the segmentation points can divide the third line segment into a plurality of sub-line segments with equal length or a plurality of sub-line segments with different lengths.

[0121] Step 204152, in response to the line segment dragging instruction, moving the plurality of segmentation points of the third line segment to target positions in the target image.

[0122] Then, the user can operate the segmentation points in the two target images displayed by the graphical user interface to send a line segment dragging instruction to the terminal device, and the terminal device moves the plurality of segmentation points of the third line segment to target positions in the target image according to the operation of the user. For example, the third line segment includes three segmentation points, and the user can drag the three segmentation points in the graphical user interface to move the three segmentation points to target positions. In one embodiment, as shown in FIG. 6, the user can drag the second segmentation point to the position of the pixel where the perpendicular point of the electric wire is located in the target image, and then drag the other two segmentation points to the positions of the pixels where the electric wire is located in the target image, so that the third line segment is divided into four sub-line segments by the three segmentation points, and the shape of the third line segment after being dragged is closer to the electric wire in the target image. Figure 11

[0123] Step 204153, calculating the seventh space coordinate and the eighth space coordinate of the top of the two electric poles in the target image according to the top positions of the two electric poles in the target image and the projection matrix of the target image.

[0124] Subsequently, the terminal device calculates the seventh space coordinate and the eighth space coordinate of the top of the two electric poles in the world coordinate system, and the process can refer to the process of calculating the first space coordinate of the top of the electric pole, which will not be described in detail in this embodiment.

[0125] Step 204154, calculating the ninth space coordinate of the segmentation point of the third line segment in the target image according to the position of the segmentation point of the third line segment in the target image and the projection matrix of the target image.

[0126] Similarly, the terminal device calculates the ninth space coordinate of each segmentation point of the third line segment in the world coordinate system according to the ninth coordinate of the pixel occupied by each segmentation point of the third line segment in the target image and the projection matrix of the target image.

[0127] Step 204155, fitting the electric wire according to the seventh space coordinate, the eighth space coordinate and the ninth space coordinate to complete the reconstruction of the electric wire.

[0128] ​Finally, the terminal device can connect the seventh space coordinate, the eighth space coordinate and each ninth space coordinate to obtain a real line segment in the world coordinate system, and perform fitting on the real line segment to complete the reconstruction of the electric wire.

[0129] On the basis of the above embodiment, in step 20415, in response to the line segment dragging instruction, the shape of the third line segment in the target image is changed, and the reconstruction of the electric wire is completed according to the third line segment after the shape is changed, the top positions of the two electric poles and the projection matrix of the target image, including:

[0130] In step 204156, in response to the line segment dragging instruction, the midpoint of the third line segment in the target image is moved to a target position.

[0131] In another embodiment, when the user changes the shape of the third line segment, the user can drag the midpoint of the third line segment in the target image displayed by the graphical user interface, so as to send a line segment dragging instruction to the terminal device, and the terminal device moves the midpoint of the third line segment in the target image to the target position dragged by the user according to the user dragging instruction. For example, as shown in Figure 12 the target position is the position of the pixel occupied by the midpoint of the electric wire in the target image, and the user can drag the midpoint of the third line segment in the target image to the position of the pixel where the perpendicular point of the electric wire is located.

[0132] In step 204157, the tenth space coordinate and the eleventh space coordinate of the top positions of the two electric poles are calculated according to the top positions of the two electric poles in the target image and the projection matrix of the target image.

[0133] Subsequently, the terminal device calculates the tenth space coordinate and the eleventh space coordinate of the top positions of the two electric poles in the world coordinate system, and the process can refer to the process of calculating the first space coordinate of the top position of the electric pole, which will not be described in detail in this embodiment.

[0134] In step 204158, the twelfth space coordinate of the midpoint of the third line segment is calculated according to the position of the midpoint of the third line segment in the target image and the projection matrix of the target image.

[0135] Similarly, the terminal device calculates the twelfth space coordinate of the midpoint of the third line segment in the world coordinate system according to the twelfth coordinate of the pixel occupied by the midpoint of the third line segment in the target image and the projection matrix of the target image.

[0136] In step 204159, the electric wire is fitted according to the tenth space coordinate, the eleventh space coordinate and the twelfth space coordinate to complete the reconstruction of the electric wire.

[0137] Finally, the terminal device can calculate a cubic spline function according to the tenth spatial coordinate, the eleventh spatial coordinate, and the twelfth spatial coordinate, and fit the electric wire according to the cubic spline function, so as to complete the reconstruction of the electric wire.

[0138] In the foregoing, the embodiment of the present application provides an obstacle reconstruction method. After a user selects a target image and selects a target obstacle type as an electric wire on a terminal device, the user can label the top positions of two electric poles by operating the target image, the terminal device generates a third line segment connecting the tops of the two electric poles, the user can drag the third line segment in the target image to make the third line segment more consistent with the actual electric wire, and finally the terminal device can calculate the spatial coordinates of the tops of the two electric poles and the spatial coordinates of the third line segment, fit the electric wire, and complete the reconstruction of the electric wire, thereby improving the reconstruction effect of the electric wire.

[0139] To further improve the accuracy of the target obstacle reconstructed by the obstacle reconstruction method in any of the foregoing embodiments, the embodiment of the present application provides another obstacle reconstruction method, as shown in Figure 13 Figure 13 The flowchart of another obstacle reconstruction method provided by the embodiment of the present application includes the following steps.

[0140] Step 301: receiving at least three images, and calculating a projection matrix of each image, wherein each image includes a target obstacle to be reconstructed.

[0141] Step 302: in response to an image selection instruction, determining a preset number of target images in the images.

[0142] Step 303: in response to an obstacle type selection instruction, determining a target obstacle type in the obstacle types, and calling a corresponding target obstacle reconstruction rule according to the target obstacle type.

[0143] Step 304: in response to an operation instruction on the target image, reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule.

[0144] Step 305: selecting a correction image from the received images, and the correction image is not the target image.

[0145] Because the user lacks a reference when labeling the target obstacle in two images, the user cannot determine whether the labeling is accurate enough. Therefore, after reconstructing the target obstacle, the position of the target obstacle in the three-dimensional space can be further corrected. When correcting, the terminal device first needs to select a correction image from the received images, and the correction image is not the target image selected by the user. ​

[0146] Step 306: Project the reconstructed target obstacle onto the corrected image to obtain the target object.

[0147] Then, the terminal device projects the reconstructed target obstacle onto the corresponding position in the corrected image based on its spatial coordinates in the world coordinate system, thus obtaining the two-dimensional target object in the corrected image. The terminal device then displays the projected corrected image. For example, such as... Figure 14 As shown, assume the user inputs three images, denoted as I1, I2, and I3, where I1 and I2 are the target images, and I3 is the correction image. The squares shown in I1 and I2 represent the true locations of the target obstacles. However, due to errors in the user's annotation of the target obstacles in I1 and I2, the reconstructed target obstacle's position deviates. Assuming the reconstructed target obstacle's spatial coordinates are P, when point P is reprojected onto I3, the generated target object in I3 is located at the position of the star shape, which significantly deviates from the position of the squares in I1 and I2. Therefore, correction is necessary.

[0148] Step 307: In response to the correction command, confirm the target position in the correction image, move the target object to the target position, and obtain the target correction image.

[0149] When a user confirms a deviation in the position of a target obstacle in the calibration image, the user can further correct the obstacle's position. Specifically, the user can use the mouse to mark the desired location of the target object in the calibration image displayed in the graphical user interface, such as... Figure 14 The square in I3. After confirmation, a correction command can be sent to the terminal device. Upon receiving the correction command, the terminal device determines the position marked by the user as the target position in the corrected image, and moves the target object to the target position, thereby obtaining the target corrected image.

[0150] Step 308: Correct the target obstacle based on the target correction image.

[0151] Finally, the terminal device corrects the spatial position of the target obstacle in three-dimensional space based on the target correction image. In one embodiment, after generating the target correction image, corresponding weights can be assigned to the target image and the target correction image, with the weight of the target correction image being greater than that of the target image. When correcting the spatial position of the target obstacle, the terminal device will combine the positions of the target obstacle in the target image and the target correction image to correct the spatial position of the target obstacle. However, since the target correction image has a higher weight, the terminal device will primarily focus on the position of the target object in the target correction image during the correction process.

[0152] In addition, it should be noted that when there are multiple target obstacles in the target object, after any one target obstacle is reconstructed, the user can reselect the target obstacle type to reconstruct other target obstacles without exiting the interactive interface until all target obstacles are reconstructed, and the user can one-key export a vectorized map including all target obstacles, as shown in Figure 15 To avoid the situation of missing labeling, the data labeled by the user is saved on the terminal device, and the user can re-call the data in the terminal device when additional labeling is needed.

[0153] The above, the embodiment of the application provides a kind of obstacle reconstruction method, after the target obstacle is reconstructed, target obstacle is further projected to the correction image, so that user can obtain whether the position of reconstructed target obstacle is accurate, if inaccurate, user can further correct the position of target obstacle, to realize more accurate obstacle reconstruction.

[0154] As shown in Figure 16 As shown in Figure 16 The structure diagram of the obstacle reconstruction device provided by the embodiment of the application includes:

[0155] The projection matrix calculation module 401 is used to receive at least two images, and calculate the projection matrix of each image, wherein each image includes a target obstacle to be reconstructed;

[0156] The image selection module 402 is used to determine a preset number of target images in the images in response to an image selection instruction;

[0157] The type selection module 403 is used to determine a target obstacle type in the obstacle types in response to an obstacle type selection instruction, and call a corresponding target obstacle reconstruction rule according to the target obstacle type;

[0158] The reconstruction module 404 is used to reconstruct the target obstacle according to the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule in response to an operation instruction on the target image.

[0159] On the basis of the above embodiment, the reconstruction module 404 includes a first coordinate calculation submodule, a second coordinate calculation submodule, a first radius determination submodule and a power pole reconstruction submodule, when the target obstacle type is a power pole:

[0160] The first coordinate calculation submodule is used to determine the bottom position of the power pole in the target image in response to a power pole bottom labeling instruction, and calculate the first space coordinate of the bottom of the power pole according to the projection matrix of the target image;

[0161] The second coordinate calculation submodule is configured to determine a top position of the utility pole in the target image in response to a utility pole top marking instruction, and calculate a second spatial coordinate of the top of the utility pole according to a projection matrix of the target image;

[0162] The first radius determination submodule is configured to determine a radius of the utility pole in response to a first radius input instruction.

[0163] The utility pole reconstruction submodule is configured to complete reconstruction of the utility pole according to the first spatial coordinate, the second spatial coordinate, and the radius of the utility pole.

[0164] On the basis of the above embodiment, the first coordinate calculation submodule is specifically configured to determine a first coordinate of a pixel occupied by a bottom position of the utility pole in the target image, and calculate a first spatial coordinate of the bottom of the utility pole according to the first coordinate and a projection matrix of the target image.

[0165] On the basis of the above embodiment, the utility pole reconstruction submodule specifically comprises:

[0166] The first line segment construction unit is configured to construct a first line segment according to the first spatial coordinate and the second spatial coordinate.

[0167] The utility pole reconstruction unit is configured to generate the utility pole according to the radius of the utility pole, with the first line segment as a central axis of the utility pole.

[0168] On the basis of the above embodiment, the reconstruction module 404 comprises a third coordinate calculation submodule, a fourth coordinate calculation submodule, and a diagonal line reconstruction submodule, when the target obstacle type is a diagonal line:

[0169] The third coordinate calculation submodule is configured to determine a bottom position of the diagonal line in the target image in response to a diagonal line bottom marking instruction, and calculate a third spatial coordinate of the bottom of the diagonal line according to a projection matrix of the target image.

[0170] The fourth coordinate calculation submodule is configured to determine a top position of the diagonal line in the target image in response to a diagonal line top marking instruction, and calculate a fourth spatial coordinate of the top of the diagonal line according to the projection matrix of the target image.

[0171] The diagonal line reconstruction submodule is configured to construct a second line segment according to the third spatial coordinate and the fourth spatial coordinate, and complete reconstruction of the diagonal line.

[0172] On the basis of the above embodiment, the reconstruction module 404 further comprises an obstacle range submodule and an obstacle generation submodule, when the number of diagonal lines is more than one:

[0173] The obstacle range submodule is configured to determine a first obstacle range according to the third spatial coordinate of each diagonal line and the fourth spatial coordinate of each diagonal line.

[0174] The obstacle generation submodule is configured to generate an obstacle according to the first obstacle range.

[0175] On the basis of the above-mentioned embodiments, the reconstruction module 404 comprises a fifth coordinate calculation submodule, a sixth coordinate calculation submodule, a second radius determination submodule, and a tree reconstruction submodule, when the target obstacle type is a tree:

[0176] The fifth coordinate calculation submodule is configured to determine the bottom position of the tree in the target image in response to the tree bottom annotation instruction, and calculate the fifth spatial coordinate of the bottom of the tree according to the projection matrix of the target image;

[0177] The sixth coordinate calculation submodule is configured to determine the top position of the tree in the target image in response to the tree top annotation instruction, and calculate the sixth spatial coordinate of the top of the tree according to the projection matrix of the target image;

[0178] The second radius determination submodule is configured to determine the radius of the tree in response to the second radius input instruction.

[0179] The tree reconstruction submodule is configured to complete the reconstruction of the tree according to the fifth spatial coordinate, the sixth spatial coordinate, and the radius of the tree.

[0180] On the basis of the above-mentioned embodiments, the second radius determination submodule is specifically configured to generate a first circle at the bottom position of the tree in the specified target image in response to the second radius input instruction, and determine the radius of the tree according to the radius of the first circle.

[0181] On the basis of the above-mentioned embodiments, the second radius determination submodule comprises:

[0182] The height determination unit is configured to determine the height of the tree according to the fifth spatial coordinate and the sixth spatial coordinate.

[0183] The ratio calculation unit is configured to determine the first pixel number occupied by the tree in the height direction in the specified target image, and determine the first ratio between the height and the first pixel number.

[0184] The tree radius calculation unit is configured to determine the second pixel number occupied by the radius of the first circle in the specified target image, multiply the second pixel number by the first ratio to obtain the radius of the tree.

[0185] On the basis of the above-mentioned embodiments, the reconstruction module 404 comprises a third line segment construction submodule and a wire reconstruction submodule, when the target obstacle type is a wire:

[0186] The third line segment construction submodule is configured to determine the top positions of the two wire poles connected to the wire in the target image in response to the wire pole top annotation instruction, and generate a third line segment connecting the top positions of the two wire poles;

[0187] The electric wire reconstruction submodule is configured to change the shape of the third line segment in the target image in response to the line segment dragging instruction, and complete the reconstruction of the electric wire according to the third line segment after the shape is changed, the top positions of the two electric poles, and the projection matrix of the target image.

[0188] On the basis of the above embodiment, the electric wire reconstruction submodule comprises:

[0189] A split point determination unit is configured to determine a plurality of split points on the third line segment in the target image, and the plurality of split points are used to divide the third line segment into a plurality of sub-line segments.

[0190] A split point dragging unit is configured to move the plurality of split points of the third line segment to target positions in the target image in response to the line segment dragging instruction.

[0191] A first coordinate calculation unit is configured to calculate a seventh space coordinate and an eighth space coordinate of the top positions of the two electric poles in the target image according to the top positions of the two electric poles in the target image and the projection matrix of the target image.

[0192] A second coordinate calculation unit is configured to calculate a ninth space coordinate of the split points of the third line segment in the target image according to the positions of the split points of the third line segment in the target image and the projection matrix of the target image.

[0193] A first electric wire reconstruction unit is configured to perform fitting of the electric wire according to the seventh space coordinate, the eighth space coordinate, and the ninth space coordinate, and complete the reconstruction of the electric wire.

[0194] On the basis of the above embodiment, the electric wire reconstruction submodule comprises:

[0195] A midpoint moving unit is configured to move the midpoint of the third line segment to a target position in the target image in response to the line segment dragging instruction.

[0196] A third coordinate calculation unit is configured to calculate a tenth space coordinate and an eleventh space coordinate of the top positions of the two electric poles in the target image according to the top positions of the two electric poles in the target image and the projection matrix of the target image.

[0197] A fourth coordinate calculation unit is configured to calculate a twelfth space coordinate of the midpoint of the third line segment in the target image according to the position of the midpoint of the third line segment in the target image and the projection matrix of the target image.

[0198] A second electric wire reconstruction unit is configured to perform fitting of the electric wire according to the tenth space coordinate, the eleventh space coordinate, and the twelfth space coordinate, and complete the reconstruction of the electric wire.

[0199] On the basis of the above embodiment, the number of received images is at least three, and the obstacle reconstruction device further comprises:

[0200] The correction image selection module is configured to select a correction image from the received images, and the correction image is not the target image.

[0201] The projection module is configured to project the reconstructed target obstacle into the correction image to obtain a target object.

[0202] The position moving module is configured to move the target object to a target position in the correction image to obtain a target correction image in response to the correction instruction and in confirmation of the target position in the correction image.

[0203] The correction module is configured to correct the target obstacle according to the target correction image.

[0204] The embodiment further provides a terminal device, as shown in Figure 17 The terminal device 50 includes a processor 500 and a memory 501.

[0205] The memory 501 is configured to store a computer program 502 and transmit the computer program 502 to the processor.

[0206] The processor 500 is configured to execute the steps of the obstacle reconstruction method in any of the above embodiments according to instructions in the computer program 502.

[0207] For example, the computer program 502 can be divided into one or more modules / units, which are stored in the memory 501 and can be executed by the processor 500 to implement the obstacle reconstruction method provided in any of the above embodiments. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 502 in the terminal device 50.

[0208] The terminal device 50 can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing devices. The terminal device 50 can include, but is not limited to, the processor 500 and the memory 501. Those skilled in the art can understand that Figure 17 The terminal device 50 is only an example and does not constitute a limitation on the terminal device 50, and can include more or fewer components than shown, or combine certain components, or different components, for example, the terminal device 50 can also include an input / output device, a network access device, a bus, etc.

[0209] The processor 500 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0210] The memory 501 can be an internal storage unit of the terminal device 50, for example, a hard disk or a memory of the terminal device 50. The memory 501 can also be an external storage device of the terminal device 50, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 50. Further, the memory 501 can also include both the internal storage unit and the external storage device of the terminal device 50. The memory 501 is used to store computer programs and other programs and data required by the terminal device 50. The memory 501 can also be used to temporarily store data that has been output or will be output.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0212] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0214] In addition, each functional unit in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0215] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various computer program storage media.

[0216] The embodiments of the present application also provide a storage medium containing computer executable instructions, which are used to execute the obstacle reconstruction method provided by any one of the embodiments of the present application when executed by a computer processor. The method comprises the following steps:

[0217] Receiving at least two images, and calculating a projection matrix of each image, wherein each image comprises a target obstacle to be reconstructed;

[0218] In response to an image selection instruction, determining a preset number of target images in the images;

[0219] In response to an obstacle type selection instruction, determining a target obstacle type in the obstacle types, and calling a corresponding target obstacle reconstruction rule according to the target obstacle type;

[0220] In response to an operation instruction on the target image, reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule.

[0221] Note that the above only describes the preferred embodiments of the present application and the applied technical principles. Those skilled in the art will understand that the embodiments of the present application are not limited to the specific embodiments described herein, and that various obvious changes, reconfigurations and substitutions can be made by those skilled in the art without departing from the scope of the embodiments of the present application. Therefore, although the embodiments of the present application have been described in detail through the above embodiments, the embodiments of the present application are not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the embodiments of the present application, and the scope of the embodiments of the present application is determined by the appended claims.

Claims

1. An obstacle reconstruction method, characterized by, The method comprises: receiving at least two images, and calculating a projection matrix of each image, wherein each image comprises a target obstacle to be reconstructed; in response to an image selection instruction, determining a preset number of target images in the images; in response to an obstacle type selection instruction, determining a target obstacle type in the obstacle types, and calling a corresponding target obstacle reconstruction rule according to the target obstacle type; in response to an operation instruction on the target images, reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule; wherein the number of received images is at least three, and the method further comprises: selecting a correction image from the received images, and the correction image is not the target image; projecting the reconstructed target obstacle into the correction image to obtain a target object; in response to a correction instruction, confirming a target position in the correction image, moving the target object to the target position to obtain a target correction image; and correcting the target obstacle according to the target correction image.

2. The obstacle reconstruction method of claim 1, wherein, The obstacle type comprises at least one of a telegraph pole, an inclined line, a tree, and a wire.

3. The obstacle reconstruction method of claim 2, wherein, When the target obstacle type is the telegraph pole, the reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule comprises: in response to a telegraph pole bottom marking instruction, determining a bottom position of the telegraph pole in the target image, and calculating a first spatial coordinate of the bottom of the telegraph pole according to the projection matrix of the target image; in response to a telegraph pole top marking instruction, determining a top position of the telegraph pole in the target image, and calculating a second spatial coordinate of the top of the telegraph pole according to the projection matrix of the target image; in response to a first radius input instruction, determining a radius of the telegraph pole; and completing the reconstruction of the telegraph pole according to the first spatial coordinate, the second spatial coordinate, and the radius of the telegraph pole.

4. The obstacle reconstruction method of claim 3, wherein, The calculating the first spatial coordinate of the bottom of the telegraph pole according to the projection matrix of the target image comprises: determining a first coordinate of a pixel occupied by the bottom position of the telegraph pole in the target image, and calculating the first spatial coordinate of the bottom of the telegraph pole according to the first coordinate and the projection matrix of the target image.

5. The obstacle reconstruction method of claim 3, wherein, The completing the reconstruction of the telegraph pole according to the first spatial coordinate, the second spatial coordinate, and the radius of the telegraph pole comprises: constructing a first line segment according to the first spatial coordinate and the second spatial coordinate; generating the telegraph pole according to the radius of the telegraph pole, with the first line segment as a central axis of the telegraph pole.

6. The method of claim 2, wherein, When the target obstacle type is the inclined line, the reconstructing the target obstacle according to the operation instruction, the projection matrix of the target image, and the target obstacle reconstruction rule comprises: determining a bottom position of the oblique line in the target image in response to the oblique line bottom annotation instruction, and calculating a third spatial coordinate of the bottom of the oblique line according to a projection matrix of the target image; determining a top position of the oblique line in the target image in response to the oblique line top annotation instruction, and calculating a fourth spatial coordinate of the top of the oblique line according to the projection matrix of the target image; constructing a second line segment according to the third spatial coordinate and the fourth spatial coordinate to complete the reconstruction of the oblique line.

7. A method of obstacle reconstruction according to claim 6, wherein, When the number of the oblique lines is more than one, the method further comprises: determining a first obstacle range according to the third spatial coordinate of each of the oblique lines and the fourth spatial coordinate of each of the oblique lines; generating an obstacle according to the first obstacle range.

8. The obstacle reconstruction method of claim 2, wherein, When the target obstacle type is the tree, the reconstruction of the target obstacle according to the operation instruction on the target image, the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule comprises: determining a bottom position of the tree in the target image in response to a tree bottom annotation instruction, and calculating a fifth spatial coordinate of the bottom of the tree according to a projection matrix of the target image; determining a top position of the tree in the target image in response to a tree top annotation instruction, and calculating a sixth spatial coordinate of the top of the tree according to the projection matrix of the target image; determining a radius of the tree in response to a second radius input instruction; completing the reconstruction of the tree according to the fifth spatial coordinate, the sixth spatial coordinate and the radius of the tree.

9. The method of claim 8, wherein, The determination of the radius of the tree in response to the second radius input instruction comprises: generating a first circle at the bottom position of the tree in a specified target image in response to a second radius input instruction; determining the radius of the tree according to the radius of the first circle.

10. The obstacle reconstruction method of claim 9, wherein, The determination of the radius of the tree according to the radius of the first circle comprises: determining a height of the tree according to the fifth spatial coordinate and the sixth spatial coordinate; determining a first pixel number occupied by the tree in a height direction in the specified target image, and determining a first ratio between the height and the first pixel number; determining a second pixel number occupied by the radius of the first circle in the specified target image, and multiplying the second pixel number by the first ratio to obtain the radius of the tree.

11. The obstacle reconstruction method of claim 2, wherein, When the target obstacle type is the electric wire, the reconstruction of the target obstacle according to the operation instruction on the target image, the operation instruction, the projection matrix of the target image and the target obstacle reconstruction rule comprises: determining top positions of two electric poles connected to the electric wire in the target image in response to an electric pole top annotation instruction to generate a third line segment connecting the top positions of the two electric poles; changing a shape of the third line segment in the target image in response to a line segment dragging instruction, and completing the reconstruction of the electric wire according to the third line segment after the shape is changed, the top positions of the two electric poles and the projection matrix of the target image.

12. The obstacle reconstruction method of claim 11, wherein, The response to the line segment dragging instruction, the shape of the third line segment in the target image is changed, and the reconstruction of the electric wire is completed according to the third line segment after shape change, the top positions of the two electric poles and the projection matrix of the target image, including: A plurality of segmentation points are determined on the third line segment of the target image, and the plurality of segmentation points are used to divide the third line segment into a plurality of sub-line segments; In response to a line segment dragging instruction, the plurality of segmentation points of the third line segment are moved to target positions in the target image; According to the top positions of the two electric poles in the target image and the projection matrix of the target image, the seventh space coordinates and the eighth space coordinates of the top positions of the two electric poles are calculated respectively; According to the positions of the segmentation points of the third line segment in the target image and the projection matrix of the target image, the ninth space coordinates of the segmentation points are calculated; The fitting of the electric wire is performed according to the seventh space coordinates, the eighth space coordinates and the ninth space coordinates, and the reconstruction of the electric wire is completed.

13. The obstacle reconstruction method of claim 11, wherein, The response to the line segment dragging instruction, the shape of the third line segment in the target image is changed, and the reconstruction of the electric wire is completed according to the third line segment after shape change, the top positions of the two electric poles and the projection matrix of the target image, including: In response to a line segment dragging instruction, the midpoint of the third line segment is moved to a target position in the target image; According to the top positions of the two electric poles in the target image and the projection matrix of the target image, the tenth space coordinates and the eleventh space coordinates of the top positions of the two electric poles are calculated respectively; According to the positions of the midpoint of the third line segment in the target image and the projection matrix of the target image, the twelfth space coordinates of the midpoint of the third line segment are calculated; The fitting of the electric wire is performed according to the tenth space coordinates, the eleventh space coordinates and the twelfth space coordinates, and the reconstruction of the electric wire is completed.

14. An obstacle reconstruction apparatus, characterized by, Including: A projection matrix calculation module is configured to receive at least two images and calculate a projection matrix of each image, wherein each image includes a target obstacle to be reconstructed; An image selection module is configured to determine a preset number of target images in the images in response to an image selection instruction; A type selection module is configured to determine a target obstacle type in an obstacle type in response to an obstacle type selection instruction, and to call a corresponding target obstacle reconstruction rule according to the target obstacle type; A reconstruction module is configured to reconstruct the target obstacle according to an operation instruction on the target image, the projection matrix of the target image and the target obstacle reconstruction rule in response to the operation instruction on the target image; Wherein, the number of received images is at least three, and the obstacle reconstruction device further comprises: A correction image selection module is configured to select a correction image from the received images, and the correction image is not the target image; A projection module is configured to project the reconstructed target obstacle into the correction image to obtain a target object; A position moving module is configured to, in response to the correction instruction, confirm a target position in the correction image, move the target object to the target position, and obtain a target correction image; A correction module is configured to correct the target obstacle according to the target correction image.

15. A terminal device, comprising: The terminal device comprises a processor and a memory; The memory is configured to store a computer program and transmit the computer program to the processor; The processor is configured to execute the obstacle reconstruction method according to the instructions in the computer program.

16. A storage medium storing computer-executable instructions, wherein: The computer executable instructions, when executed by the computer processor, are configured to execute the obstacle reconstruction method according to any one of claims 1-13.

Images