A method, system, device and storage medium for checking intersection points

Abstract

The present invention discloses an intersection point verification method, which is applied to a mobile terminal. The method comprises: verifying whether a current target intersection point is qualified; when it is verified that the current target intersection point is unqualified, controlling a drone to fly within a preset range of the target intersection point and acquiring a regional image of the preset range; determining a calibration intersection point based on the regional image, making a traditional manual field verification method intelligent, saving manpower output, and improving the efficiency of intersection point verification; when it is verified that the current target intersection point is unqualified, calibrating the intersection point to ensure that the location of the calibrated intersection point meets the construction requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling and positioning, and in particular to an intersection point verification method, system, device and storage medium. Background Art

[0002] In the process of transmission line engineering pole tower construction, pole towers are needed. Before the pole tower construction, it is necessary to determine the coordinate position of the intersection point and whether the surrounding environment of the intersection point is suitable for setting up the pole tower. If the actual geographical environment of the intersection point is not suitable for setting up the pole tower, it will affect the construction, delay the construction time, and increase the construction cost.

[0003] Currently, the most commonly used method for checking intersection points is for operators to take drawings that have been pre-made based on high-definition images and other information to the field for verification. The operators look for the corresponding intersection points based on the drawings for verification and determine whether the intersection points indicated on the drawings meet the actual conditions for erecting poles. After the verification is completed, the verification results are manually sent back.

[0004] In cities, it is relatively easy to find intersection points based on drawings because there are various reference objects such as buildings. However, in the wild environment where there are no reference objects, it is quite difficult to rely on drawings to find the locations of intersection points. Moreover, when manual verification of intersection points relies on maps, it is time-consuming and inefficient. In addition, the wild environment is complex and the verification process is somewhat dangerous. Summary of the invention

[0005] The present invention provides a method, system, device and storage medium for checking intersection points, so as to solve the problem of long time consumption and low efficiency when manually checking intersection points according to drawings, and obtain appropriate and accurate intersection point coordinate positions when the intersection point positions on the drawings need to be updated.

[0006] According to one aspect of the present invention, a method for checking an intersection point is provided, the method being applied to a mobile terminal, the method comprising:

[0007] Check whether the current target intersection point is qualified;

[0008] When it is verified that the current target intersection point is unqualified, controlling the UAV to fly within a preset range of the target intersection point and acquiring an area image of the preset range;

[0009] A calibration intersection point is determined based on the area image.

[0010] According to one aspect of the present invention, a method for checking intersection points is provided, the method being applied to a server, the method comprising:

[0011] Acquire the calibration intersection point image collected by the UAV for the calibration intersection point;

[0012] Performing three-dimensional reconstruction on the calibration intersection points based on the calibration intersection point images to obtain a three-dimensional model;

[0013] The coordinate position of the calibration intersection point is determined according to the three-dimensional model.

[0014] According to another aspect of the present invention, a crossing point verification system is provided, the system comprising a mobile terminal, a drone and a server, wherein:

[0015] The mobile terminal is used to: check whether the current target intersection point is qualified, and when the current target intersection point is found to be unqualified, control the UAV to fly within a preset range of the target intersection point and obtain a regional image of the preset range, and determine a calibration intersection point based on the regional image;

[0016] The server is used to: obtain the calibration intersection point image collected by the drone for the calibration intersection point, and perform three-dimensional reconstruction of the calibration intersection point based on the calibration intersection point image to obtain a three-dimensional model, and determine the coordinate position of the calibration intersection point according to the three-dimensional model.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform an intersection point verification method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement an intersection point verification method described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention provides an intersection point verification method, which is applied to a mobile terminal to verify the current target intersection point to determine whether the target intersection point determined in the planning period meets the actual construction conditions. When it is verified that the current target intersection point is unqualified, the drone is controlled to fly within a preset range of the target intersection point and obtain a regional image of the preset range. The calibration intersection point is determined based on the regional image, and the traditional manual field verification method is intelligent, which saves manpower output and improves the efficiency of intersection point verification. When it is verified that the current target intersection point is unqualified, the intersection point is calibrated to ensure that the location of the calibrated intersection point meets the construction requirements.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a method for checking intersection points provided according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flow chart of a method for checking intersection points provided according to Embodiment 2 of the present invention;

[0027] Figure 3 is a schematic diagram of collecting and calibrating intersection point images by a drone according to Embodiment 2 of the present invention;

[0028] Figure 4 This is a schematic diagram of a cross-point verification system provided according to Embodiment 3 of the present invention;

[0029] Figure 5 This is a schematic diagram of the workflow of a cross-point verification system provided according to Embodiment 3 of the present invention;

[0030] Figure 6 It is a structural schematic diagram of an electronic device for implementing an intersection point verification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 A flowchart of a method for checking an intersection point is provided for the first embodiment of the present invention, and the method is applied in a mobile terminal.

[0035] The mobile terminal can control the drone, receive real-time pictures and images collected by the drone, and communicate with the server through the wireless network.

[0036] When planning intersection points, planners can use some map tools with high-definition images and drones to collect preliminary images of the construction area, thereby building a preliminary model of the construction site and determining the coordinates of the target intersection points based on the preliminary model. The accuracy of the preliminary model established in this way is generally at the meter level, so the coordinates of the target intersection points determined based on the preliminary model may have some factors that do not meet the requirements of being intersection points in the actual geographical environment.

[0037] In addition, it should be clear that the coordinate position of the intersection point mentioned in this embodiment refers to the use of a point coordinate to represent the position of the intersection point. In fact, the intersection point is a circular surface with a corresponding radius range. Generally speaking, the coordinate position of the intersection point can be the coordinate position of the center of the corresponding circular surface.

[0038] The intersection point verification method provided by the embodiment of the present invention can be used not only in the pole tower construction mentioned in the background technology, but also in other industries where there is a need for intersection point verification. The method can be executed by an intersection point verification device, which can be implemented in the form of hardware and / or software.

[0039] like Figure 1 As shown, the method comprises the following steps

[0040] S110, checking whether the current target intersection point is qualified, if not, executing S120.

[0041] For each pole tower construction site, due to the structural characteristics of the pole tower, there will be multiple target intersection points in a construction area. When checking the target intersection points, they can be checked one by one.

[0042] When checking the target intersection point, you are actually checking the certain spatial range where the target intersection point is located. Since different types of poles and towers have different requirements for the size and shape of the intersection point, there are corresponding standards for judging whether the target intersection point is qualified. For example, a cement pole tower needs to meet the construction conditions within 4 meters around the intersection point coordinate position, while a high-voltage pole tower needs to meet the construction conditions within 6 to 15 meters around the intersection point coordinate position.

[0043] The reason why the target intersection point is unqualified may be that the environment within the specified area around the target intersection point does not meet the construction conditions. For example, the actual geographical environment within the range includes paddy fields, rocks, and excessive height differences in the surrounding areas.

[0044] When checking, the mobile terminal can use image recognition technology to obtain images collected by drones within a certain range for target intersection points for analysis, and identify factors that do not meet construction conditions in the images, such as rocks and paddy fields. If the identification result shows that these factors do not meet construction conditions exist, the current target intersection point can be considered unqualified. If these factors do not meet construction conditions do not exist, the current target intersection point can be considered qualified, the coordinate position of the current target intersection point is confirmed, and then the next target intersection point is checked.

[0045] In one embodiment, S110 includes the following steps:

[0046] S110-1, obtaining an intersection point image collected by the drone for the target intersection point to be checked;

[0047] S110-2, displaying the intersection point image;

[0048] S110-3, receiving the user's instruction information, if the instruction information is to confirm the coordinate position of the current target intersection point, then the current target intersection point is determined to be qualified, if the instruction information is to calibrate the coordinate position of the current target intersection point, then the current target intersection point is determined to be unqualified.

[0049] The coordinate position of the target intersection point can be determined by planners using tools such as maps containing high-definition images and drones to conduct preliminary image collection of the construction area. The mobile terminal can control the drone to take the position of the specified height above the coordinate position of the target intersection point to be verified as the end point of the flight route, and go to shoot the target intersection point to be verified, so that the mobile terminal can obtain the intersection point image collected by the drone for the target intersection point to be verified. The intersection point image collected for the target intersection point mentioned here includes a certain range including the target intersection point. For example, the environment within 4 meters around the coordinate position of the intersection point of the cement tower needs to meet the construction conditions. When collecting the intersection point image for the target intersection point of the cement tower, it can be an image within 4 meters around the target intersection point.

[0050] In addition, the mobile terminal can also first obtain the automatic flight route designed and generated for the target intersection point to be verified. The mobile terminal can directly issue an order to the drone to perform automatic flight according to the automatic flight route. Compared with directly controlling the drone to fly to the target intersection point with the mobile terminal and needing to find the target intersection point during the flight, this method can more efficiently and quickly reach the specified altitude above the target intersection point coordinate position, thereby collecting the intersection point image for the target intersection point to be verified.

[0051] After the mobile terminal acquires the intersection point image, the configured display module can display the intersection point image for the user to view. The user makes a judgment on whether the intersection point image is qualified based on the displayed intersection point image. After the judgment is completed, the user can send the instruction information on the mobile terminal. The mobile terminal receives the user's instruction information. If the instruction information is to confirm the coordinate position of the current target intersection point, the current target intersection point is determined to be qualified. If the instruction information is to calibrate the coordinate position of the current target intersection point, the current target intersection point is determined to be unqualified.

[0052] S120, controlling the UAV to fly within a preset range of the target intersection point and acquiring an area image within the preset range.

[0053] When the verification result shows that the current target intersection point is unqualified, it is necessary to re-determine a qualified point within the preset range of the current target intersection point to replace the unqualified target intersection point. The re-determined point is called a calibration intersection point. For example, when the target intersection point is unqualified, the mobile terminal can control the drone to fly within a preset range to find a suitable intersection point position near the target intersection point. The preset range can be a range with the unqualified target intersection point as the center and a radius of 30 meters. During the flight, the drone collects regional images of the preset range.

[0054] S130: Determine a calibration intersection point based on the regional image.

[0055] In the same way that the mobile terminal can use image recognition technology to determine whether the target intersection point is qualified during the verification, the mobile terminal can use image recognition technology to analyze the regional image and identify the factors that do not meet the conditions for tower construction in the regional image, such as rocks, paddy fields, etc. It should be noted that since the preset range of regional image acquisition is a relatively large range relative to the intersection point, when determining the calibration intersection point, the local area in the regional image indicating a certain area size that meets the range requirements of the intersection point and the environmental factors that meet the construction conditions can be confirmed as the area where the calibration intersection point is located. The calibration intersection point can be determined in the local area according to a certain strategy, for example, the center point of the local area can be used as the calibration intersection point, or the calibration intersection point can be determined by combining certain geographical conditions that are conducive to construction, for example, a point as far away from the road or residential area as possible in the local area can be determined as the calibration intersection point.

[0056] In one embodiment, S130 includes the following steps:

[0057] S130-1, display area image;

[0058] S130-2, detecting a calibration intersection point determination operation, the calibration intersection point determination operation is initiated by the user after determining the calibration intersection point according to the displayed regional image;

[0059] S130-3, determining a calibration intersection point according to a calibration intersection point determination operation.

[0060] The calibration intersection point can be manually determined by the user, which reduces the hardware configuration requirements for the mobile terminal to perform image recognition and image judgment, while ensuring that the process of determining the calibration intersection point based on the regional image is completed accurately and quickly.

[0061] After the drone collects the regional image, the regional image can be displayed through the mobile terminal, and the user can view the regional image through the mobile terminal and determine the calibration intersection point from the regional image according to the viewing situation. When the user determines the calibration intersection point, he can determine a local area that meets the conditions from the regional image and specify a point in the local area as the calibration intersection point.

[0062] After the user determines the calibration intersection point by viewing the regional image in the mobile terminal, the calibration intersection point determination operation can be initiated in the mobile terminal, so that after the mobile terminal receives the calibration intersection point determination operation, the calibration intersection point can be determined and the relevant information can be sent to the server. Exemplarily, the mobile terminal is configured with a touch screen, and the user performs a specified operation on a certain point in the local area of ​​the regional image in the touch screen, such as double-clicking the point. When the mobile terminal detects the double-click operation on a certain point, it is considered that the user has issued a calibration intersection point determination operation for the point, and the point is determined as the calibration intersection point.

[0063] In an embodiment of the present invention, a method for checking intersection points is disclosed. The method is applied to a mobile terminal to check a current target intersection point to determine whether the target intersection point determined in the planning period meets the actual construction conditions. When it is verified that the current target intersection point is unqualified, a drone is controlled to fly within a preset range of the target intersection point and obtain a regional image of the preset range. A calibration intersection point is determined based on the regional image, and the traditional manual field verification method is intelligentized, which saves manpower output and improves the efficiency of intersection point verification. When it is verified that the current target intersection point is unqualified, the intersection point is calibrated to ensure that the location of the calibrated intersection point meets the construction requirements.

[0064] Embodiment 2

[0065] Figure 2 A flowchart of a method for checking intersection points is provided for the second embodiment of the present invention, and the method is applied to a server.

[0066] The server can receive images, videos and other data information uploaded by drones, and can also communicate with mobile terminals through wireless networks.

[0067] When planning intersection points, the preliminary model established by the planner and information about the target intersection points, such as the number of target intersection points in a construction area, the coordinate positions of the target intersection points in the preliminary model, etc., can be sent to the server in advance. When the user carries the mobile terminal and the drone to the vicinity of the construction area, the mobile terminal can obtain the information of the corresponding construction area from the server for subsequent verification operations.

[0068] Since the UAV itself has errors in its positioning, and the UAV does not collect images for a certain point, but instead shoots a certain range based on a certain point, when the specific coordinate position of the intersection point is accurately determined, a local model can be constructed for the local area in the construction area, and the precise coordinate position of the intersection point can be determined through the local model.

[0069] like Figure 2 As shown, the method comprises the following steps:

[0070] S210, obtaining a calibration intersection point image collected by the drone for the calibration intersection point.

[0071] When the UAV collects the calibration intersection point image for the calibration intersection point, you can refer to Figure 3 A schematic diagram of a drone collecting calibration intersection point images shows that the drone can hover over the calibration intersection point, for example, at a height of 10m to 30m above the ground, and then collect images within the specified range of the calibration intersection point to obtain the calibration intersection point image. For example, the cement tower needs to meet the environmental conditions within 4 meters around the intersection point coordinate position, and the specified range area can be a circular area with a radius of 4 meters and a center point directly below the drone.

[0072] There may be multiple images of the calibration intersection point collected, and when the drone is shooting, the camera of the drone may shoot at different viewing angles, for example, 9 images may be taken from 9 directions, namely, directly below the drone, in front, behind, left, right, left front, right front, left rear, and right rear. Among them, in order to achieve higher accuracy when performing three-dimensional reconstruction based on the area within a specified range where the calibration intersection point is located and to obtain more accurate coordinate positions of the calibration intersection point, a certain overlap rate may be set between every two images, for example, an overlap rate of 30% to 50% between every two images.

[0073] S220, performing three-dimensional reconstruction on the calibration intersection points based on the calibration intersection point images to obtain a three-dimensional model.

[0074] While obtaining multiple images of the calibration intersection points, the camera angle and other information of the drone can also be obtained to perform 3D reconstruction of the specified range of the calibration intersection points to obtain a 3D model of the area. After obtaining the 3D model, the coordinate position of any point in the 3D model can be determined, so the coordinate position of the calibration intersection points can be accurately determined.

[0075] In one embodiment, S220 includes the following steps:

[0076] S220-1, extracting feature points of the calibration intersection point image.

[0077] When extracting the feature points of the calibration intersection point image, the GPU (Graphics Processing Unit) accelerated version of the SIFT algorithm (Scale-invariant feature transform) can be used to extract the feature points of each calibration intersection point image. The SIFT algorithm itself has good robustness, good stability and invariance, can adapt to changes in rotation, scale scaling, and brightness, and can be unaffected by changes in perspective, affine transformation, and noise to a certain extent.

[0078] S220-2, matching the feature points to obtain multiple groups of matching points, and determining the pixel coordinates of the matching points in the calibration intersection point image.

[0079] When matching feature points, you can select any two images from all the calibration intersection point images for matching each time. When matching, you can use the Euclidean distance. For example, you can select any two calibration intersection point images, record them as Figure A and Figure B, take a feature point in Figure A, and find the first two key points with the closest Euclidean distance to Figure B. Among these two key points, if the closest distance divided by the next closest distance is less than a certain ratio threshold, the key point with the closest distance in the current Figure B is marked as a candidate matching point for the feature point in Figure A. Process the remaining feature points in this way until all matching points in Figure A are found. In this way, the feature points of the overlapping image parts in all calibration intersection point images can be matched to obtain candidate matching points, and the pixel coordinates of the candidate matching points in the calibration intersection point images can be determined.

[0080] Errors may occur when determining candidate matching points, resulting in some candidate matching points being actually erroneous matching points. In order to improve the accuracy of 3D reconstruction, erroneous matching points need to be eliminated.

[0081] When eliminating incorrectly matched candidate matching points, the RANSAC algorithm (Random Sample Consensus) can be used. Before using the RANSAC algorithm to determine incorrect candidate matching points, several groups of candidate matching points can be randomly selected to establish an estimation model, and the remaining candidate matching points can be evaluated using this estimation model. Points that meet the model are considered to be in-game points, and those that do not meet the model are "out-game points" (out-game points are incorrectly matched candidate matching points). The in-game points are then input into the estimation model, and the error rate of the estimation model is calculated. After multiple iterations, the estimation model with the lowest error rate is used as the final model for evaluating whether the candidate matching points are incorrectly matched. All candidate matching points can be input into the final estimation model, and all out-game points are considered to be incorrectly matched. The incorrectly matched candidate matching points are eliminated, and the correct matching points are left.

[0082] S220-3, based on the camera parameters and pixel coordinates of the UAV, determine the coordinates of the matching points in the world coordinate system.

[0083] When performing 3D reconstruction, it is necessary to determine the coordinates of the matching points in the world coordinate system. The coordinates of the matching points in the world coordinate system can be calculated by solving the projection equations. The projection equations can be constructed using the camera posture, coordinate parameters and homonymous points of the drone. A homonymous point refers to the projection of the same physical location in the scene corresponding to the pixels of two or more different photos. The homonymous points can also be used to calibrate the image, converting each point in the calibration intersection point image from its own pixel coordinates to the same coordinate system for subsequent calculations.

[0084] S220-4, using the coordinates of the matching points in the world coordinate system, establish a three-dimensional skeleton.

[0085] The coordinates of the matching points in the world coordinate system can be used through the irregular triangulated network model to complete the establishment of the three-dimensional skeleton.

[0086] S220-5, determining the coordinates of the pixel points other than the feature points in the calibration intersection point image in the world coordinate system as supplementary coordinates.

[0087] In order to obtain a more accurate 3D model, the non-feature point information in the calibration intersection point image can be superimposed on the 3D skeleton. When determining the supplementary coordinates, the calibration intersection point image can be directly input into the incremental motion structure recovery algorithm, and the algorithm completes the determination of the supplementary coordinates.

[0088] S220-6, perform three-dimensional reconstruction based on the three-dimensional skeleton and supplementary coordinates to obtain a three-dimensional model.

[0089] The three-dimensional skeleton is also input into the incremental motion structure recovery algorithm. The three-dimensional skeleton contains the coordinates of the matching points, which can be the coordinates in the same coordinate system after the image is calibrated with the same name points. Together with the supplementary coordinates and camera parameters, such as camera focal length, camera position and other information, the incremental motion structure recovery algorithm is used to complete the three-dimensional reconstruction and obtain the three-dimensional model.

[0090] In the 3D model, the distance from the feature point to the center of the two cameras in the drone (the camera positions are different when taking different photos) can be used to obtain the depth information of the feature point. After obtaining the depth of the feature point, combined with the camera shooting angle, camera position (coordinate position in the world coordinate system) and other parameters, the coordinate position of the feature point in the world coordinate system can be obtained from the 3D model.

[0091] Theoretically, the coordinate position in the world coordinate system can be obtained. However, in the actual calculation process, the calculated coordinate position of each feature point in the world coordinate system is not completely consistent with the real position relationship, and the coordinates need to be optimized. In this embodiment, the coordinate position of each feature point in the world coordinate system can be optimized by minimizing the square sum of the errors between the image projection point of the object space three-dimensional point and the corresponding observation image point.

[0092] S230, determining the coordinate position of the calibration intersection point according to the three-dimensional model.

[0093] When determining the calibration intersection point in the three-dimensional model, you can first obtain the location information where the drone stops when shooting, and use the location directly below as the calibration intersection point. You can also receive the calibration intersection point selected by the backend user in the three-dimensional model. Since the coordinate position of any point can be determined in the three-dimensional model, the coordinate position of the calibration intersection point can be directly obtained after determining the calibration intersection point in the three-dimensional model.

[0094] In one embodiment, S230 includes the following steps:

[0095] The coordinate position of the center point of the 3D model is used as the coordinate position of the calibration intersection point.

[0096] In order to more quickly and conveniently determine the calibration intersection point from the three-dimensional model, the coordinate position of the center point of the three-dimensional model can be directly used as the coordinate position of the calibration intersection point.

[0097] In one embodiment, the following steps are also included:

[0098] Receive the coordinate position of the target intersection point to be checked;

[0099] Generate the automatic flight route of the UAV based on the coordinate position of the target intersection point to be verified;

[0100] The automatic flight route is sent to the mobile terminal, so that the mobile terminal controls the UAV to fly to the location of the target intersection point based on the automatic flight route.

[0101] When checking the target intersection point, the drone can fly automatically and directly go to the target intersection point to be checked. At this time, the automatic flight route followed by the drone during automatic flight can be generated by the server.

[0102] When the server generates the automatic flight route, the coordinate position of the target intersection point to be checked can be used as the end point of the automatic flight route. The target intersection point to be checked can be uploaded to the server in advance. The automatic flight route generated based on the target intersection point can ensure that the position of the target intersection point is accurately found, and the positioning accuracy of the verification is high.

[0103] When the drone conducts a preliminary flight survey of the construction area during the intersection planning period, it can obtain oblique photography data, such as the height and angle of some objects in the construction area, which can also be uploaded to the server in advance.

[0104] The server can generate an automatic flight route for the drone based on the coordinate position of the target intersection point to be checked and in combination with the oblique photography data. After the server generates the automatic flight route, when the mobile terminal needs to check the target intersection point to be checked, the server can send the corresponding automatic flight route to the mobile terminal, so that the mobile terminal can control the drone to fly to the location of the target intersection point based on the automatic flight route.

[0105] An embodiment of the present invention provides an intersection point verification method, which is applied to a server, obtains a calibration intersection point image collected by a drone for the calibration intersection point, performs three-dimensional reconstruction of the calibration intersection point based on the calibration intersection point image, obtains a three-dimensional model, and determines the coordinate position of the calibration intersection point according to the three-dimensional model. By performing three-dimensional reconstruction on the calibration intersection point, an accurate coordinate position of the calibration intersection point that meets construction conditions can be obtained, which is beneficial for subsequent use of the accurate coordinate position for construction and other operations. In addition, the verification process can be reviewed based on the calibration intersection point image, thereby ensuring the reliability of the verification.

[0106] Embodiment 3

[0107] Figure 4 This is a schematic diagram of a cross-point verification system provided by Embodiment 3 of the present invention. Figure 4 As shown, the system includes a mobile terminal, a drone and a server, wherein:

[0108] The mobile terminal is used to: check whether the current target intersection point is qualified, and when it is checked that the current target intersection point is unqualified, control the UAV to fly within a preset range of the target intersection point and obtain a regional image of the preset range, and determine the calibration intersection point based on the regional image;

[0109] The server is used to: obtain the calibration intersection point image collected by the drone for the calibration intersection point, and perform three-dimensional reconstruction of the calibration intersection point based on the calibration intersection point image to obtain a three-dimensional model, and determine the coordinate position of the calibration intersection point according to the three-dimensional model.

[0110] In this embodiment, the intersection point verification system is used to illustrate the interactive operations between the mobile terminal, the drone, and the server, so that the embodiment can be more clearly understood.

[0111] refer to Figure 5A workflow diagram of an intersection point verification system. When a user brings a mobile terminal and a drone to the vicinity of a construction area, he can log in to the corresponding APP application through the mobile terminal and confirm the verification task corresponding to the construction area. The server can generate an automatic flight route for each target intersection point to be verified by uploading the location of the target intersection point, preliminary model and other information in the initial planning stage in advance. The drone can accurately go to the current target intersection point to be verified according to the automatic flight route, and the mobile terminal can display the coordinate information of the drone throughout the operation process, so that the user can confirm the location of the drone at any time.

[0112] When the UAV is over the current target intersection point, it can collect intersection point images for the current target intersection point to be verified. The collected intersection point images are displayed through the mobile terminal, and the user makes a judgment based on the geographical environment of the current target intersection point. If the geographical environment meets the construction conditions, the target intersection point is judged to be qualified; if the geographical environment does not meet the construction conditions, the target intersection point is judged to be unqualified.

[0113] If the current target intersection point is qualified, the user can confirm and mark the target intersection point on the mobile terminal, and then check the next target intersection point to be checked.

[0114] If it is verified that the current target intersection point is unqualified, the UAV can be converted from automatic flight to mobile terminal controlled flight. The mobile terminal controls the UAV to fly within a preset range near the target intersection point and obtains the regional image of the preset range, and displays the regional image. The user can determine the calibration intersection point on the mobile terminal based on the regional image. At this point, the determined calibration intersection point is only a point on the regional image, and its specific coordinate position cannot be determined.

[0115] After determining the calibration intersection point, the mobile terminal can control the UAV to collect the calibration intersection point image for the calibration intersection point and upload it to the server. After the server obtains the calibration intersection point image, it combines the previously uploaded target intersection point position, preliminary model and other information in the initial planning stage, and establishes a three-dimensional model for the local area where the calibration intersection point is located through the aerial triangulation modeling method. The calibration intersection point can be selected in the three-dimensional model, or the center point of the three-dimensional model can be set as the calibration intersection point to obtain the coordinate position of the calibration intersection point in the world coordinate system. The server can mark the coordinate position of the calibration intersection point and replace the corresponding unqualified target intersection point. After completion, the next target intersection point to be verified can be verified.

[0116] The embodiment of the present invention discloses a cross-pile point verification system, which includes a mobile terminal, a drone and a server, wherein the mobile terminal is used to: verify whether the current target cross-pile point is qualified, when the current target cross-pile point is verified to be unqualified, control the drone to fly within a preset range of the target cross-pile point and obtain a regional image of the preset range, determine the calibration cross-pile point based on the regional image, and make the traditional manual field verification method intelligent, save manpower output, improve the efficiency of cross-pile point verification, calibrate the cross-pile point, and ensure that the location of the calibration cross-pile point meets the construction requirements. The server is used to: obtain the calibration cross-pile point image collected by the drone for the calibration cross-pile point, and perform three-dimensional reconstruction of the calibration cross-pile point based on the calibration cross-pile point image to obtain a three-dimensional model, and determine the coordinate position of the calibration cross-pile point according to the three-dimensional model, so as to obtain the accurate coordinate position of the calibration cross-pile point that meets the construction conditions, which is conducive to the subsequent use of the accurate coordinate position for construction and other operations, and in addition, the verification process can be reviewed according to the calibration cross-pile point image, so as to ensure the reliability of the verification.

[0117] The intersection point verification system provided by the embodiment of the present invention can implement the intersection point verification method provided by the first embodiment of the present invention and the second embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0118] Embodiment 4

[0119] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 6As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a cross-point verification method.

[0123] In some embodiments, a cross-stake verification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cross-stake verification method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a cross-stake verification method in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for checking intersection points, characterized in that: The method is applied to a mobile terminal, and the method comprises: Checking whether the current target intersection point is qualified, including: obtaining an intersection point image collected by a drone for the current target intersection point to be checked; displaying the intersection point image; receiving instruction information from a user, if the instruction information is to confirm the coordinate position of the current target intersection point, then determining that the current target intersection point is qualified; if the instruction information is to calibrate the coordinate position of the current target intersection point, then determining that the current target intersection point is unqualified; When it is verified that the current target intersection point is unqualified, controlling the UAV to fly within a preset range of the target intersection point and acquiring an area image of the preset range; Determining a calibration intersection point based on the area image includes: displaying the area image; detecting a calibration intersection point determination operation, wherein the calibration intersection point determination operation is initiated by the user after the user determines the calibration intersection point based on the displayed area image; and determining the calibration intersection point based on the calibration intersection point determination operation.

2. A method for checking intersection points, characterized in that: The method is applied to a server, and the method comprises: Acquire the calibration intersection point image collected by the UAV for the calibration intersection point; Performing three-dimensional reconstruction on the calibration intersection points based on the calibration intersection point images to obtain a three-dimensional model; Determine the coordinate position of the calibration intersection point according to the three-dimensional model; Wherein, before obtaining the calibration intersection point image collected by the drone for the calibration intersection point, the mobile terminal verifies whether the current target intersection point is qualified, including: obtaining the intersection point image collected by the drone for the current target intersection point to be verified; displaying the intersection point image; receiving the user's instruction information, if the instruction information is to confirm the coordinate position of the current target intersection point, then determining that the current target intersection point is qualified, if the instruction information is to calibrate the coordinate position of the current target intersection point, then determining that the current target intersection point is unqualified; When it is verified that the current target intersection point is unqualified, controlling the UAV to fly within a preset range of the target intersection point and acquiring an area image of the preset range; Determining a calibration intersection point based on the area image includes: displaying the area image; detecting a calibration intersection point determination operation, wherein the calibration intersection point determination operation is initiated by the user after the user determines the calibration intersection point based on the displayed area image; and determining the calibration intersection point based on the calibration intersection point determination operation.

3. The method according to claim 2, characterized in that The three-dimensional reconstruction of the calibration intersection point based on the calibration intersection point image to obtain a three-dimensional model includes: Extracting feature points of the calibration intersection point image; Matching the feature points to obtain multiple groups of matching points, and determining pixel coordinates of the matching points in the calibration intersection point image; Determine the coordinates of the matching point in a world coordinate system based on the camera parameters of the drone and the pixel coordinates; Using the coordinates of the matching points in the world coordinate system, a three-dimensional skeleton is established; Determine the coordinates of the pixel points other than the feature points in the calibration intersection point image in the world coordinate system as supplementary coordinates; Three-dimensional reconstruction is performed according to the three-dimensional skeleton and the supplementary coordinates to obtain the three-dimensional model.

4. The method according to claim 2 or 3, characterized in that: Determining the coordinate position of the calibration intersection point according to the three-dimensional model includes: The coordinate position of the center point of the three-dimensional model is used as the coordinate position of the calibration intersection point.

5. The method according to claim 2, characterized in that: The method further comprises: Receive the coordinate position of the target intersection point to be checked; Based on the coordinate position of the target intersection point to be checked, an automatic flight route of the UAV is generated; The automatic flight route is sent to a mobile terminal, so that the mobile terminal controls the unmanned aerial vehicle to fly to the location of the target intersection point based on the automatic flight route.

6. A cross-stake point verification system, characterized in that: The system includes a mobile terminal, a drone and a server, wherein: The mobile terminal is used to: check whether the current target intersection point is qualified, including: obtaining the intersection point image collected by the drone for the current target intersection point to be checked; displaying the intersection point image; receiving the user's instruction information, if the instruction information is to confirm the coordinate position of the current target intersection point, then the current target intersection point is determined to be qualified, if the instruction information is to calibrate the coordinate position of the current target intersection point, then the current target intersection point is determined to be unqualified; when it is verified that the current target intersection point is unqualified, the drone is controlled to fly within a preset range of the target intersection point and obtain a regional image of the preset range; based on the regional image, a calibration intersection point is determined, including: displaying the regional image; detecting a calibration intersection point determination operation, the calibration intersection point determination operation is initiated by the user after determining the calibration intersection point according to the displayed regional image; determining the calibration intersection point according to the calibration intersection point determination operation; The server is used to: obtain the calibration intersection point image collected by the drone for the calibration intersection point, and perform three-dimensional reconstruction of the calibration intersection point based on the calibration intersection point image to obtain a three-dimensional model, and determine the coordinate position of the calibration intersection point according to the three-dimensional model.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for checking intersection points according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement an intersection point verification method according to any one of claims 1 to 5 when executed.

Images