Navigation route planning method and apparatus for three-dimensional modeling, and electronic device and storage medium

Abstract

The present application relates to the technical field of navigation route planning. Provided are a navigation route planning method and apparatus for three-dimensional modeling, and an electronic device and a storage medium. The method comprises: acquiring point cloud data corresponding to a region of interest; on the basis of an image overlap rate, cutting the point cloud data along a target reference axis, so as to obtain a plurality of target point cloud sections, and capturing from the point cloud data a point cloud annulus corresponding to each target point cloud section; controlling a pre-constructed collision sphere to move around each target point cloud section, so as to determine, on the basis of the collision between the collision sphere and each target point cloud section and each point cloud annulus during movement, an envelope line corresponding to each target point cloud section; and on the basis of points included in each envelope line, generating an image collection navigation route corresponding to the region of interest. The embodiments of the present application can significantly improve the precision and the degree of automation of navigation route planning, and avoid the occurrence of a collision accident to an unmanned aerial vehicle, thereby facilitating an improvement to the precision of subsequent three-dimensional modeling, and reducing potential safety hazards that are present during the navigation of the unmanned aerial vehicle.

Description

Three-dimensional modeling route planning method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application number 2023115764619 filed with the Chinese Patent Office on November 24, 2023, entitled “Route planning method, device, electronic device and storage medium for three-dimensional modeling”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of route planning, and in particular to a three-dimensional modeling route planning method, device, electronic device and storage medium. Background Art

[0004] When detailed 3D modeling of a scene is required, a detailed scan is often performed. This involves using photographic equipment to capture photos of all visible surfaces of the scene, ensuring a certain overlap ratio. This process is often performed using drones, but drones require manual flight planning, which is inefficient.

[0005] At present, in order to improve the above-mentioned problems, the relevant technology proposes "A drone shooting and reconstruction method and device for fine modeling of building facades". This technology uses the camera on a drone to shoot the image of the building vertically downward from the air. After photogrammetric processing, the plane and elevation information of the building are obtained, the outer contour range of the building is selected, and a rough three-dimensional framework model of the building is constructed in combination with the elevation information; the floor plan of the building is extracted based on the rough model of the building, and the outer contour line of the building is obtained by establishing the outer boundary of the building floor plan. Based on the outer contour line of the building, the horizontal or vertical drone flight path of the building facade is generated. However, because this related technology uses a rough three-dimensional framework model of the building to generate the drone flight path, there will be problems such as low accuracy of the drone flight path and a high probability of collision between the drone and the building. This not only affects the accuracy of the subsequent three-dimensional modeling, but also poses certain safety risks in the process of drone image acquisition.

[0006] Summary of the Invention

[0007] In view of this, the purpose of this application includes, for example, providing a three-dimensional modeling route planning method, device, electronic device and storage medium to significantly improve the accuracy and automation of planned routes, avoid drone collision accidents, thereby helping to improve the accuracy of subsequent three-dimensional modeling, and reduce safety hazards during drone navigation.

[0008] In a first aspect, an embodiment of the present application provides a three-dimensional modeling route planning method, comprising:

[0009] Get the point cloud data corresponding to the area of ​​interest;

[0010] Based on a preset image overlap rate, the point cloud data is cut along a target reference axis to obtain a plurality of target point cloud sections, and a point cloud ring corresponding to each target point cloud section is intercepted from the point cloud data;

[0011] Controlling a pre-constructed collision sphere to move around the target point cloud section, so as to determine an envelope corresponding to the target point cloud section according to collision conditions between the collision sphere, the target point cloud section, and the point cloud annulus during the movement;

[0012] An image acquisition route corresponding to the region of interest is generated based on the points included in the envelope.

[0013] Optionally, based on a preset image overlap ratio, the step of cutting the point cloud data along a target reference axis to obtain a plurality of target point cloud sections includes:

[0014] Determining at least one target reference axis from coordinate axes included in the coordinate system corresponding to the region of interest;

[0015] Determining a horizontal spacing between images and a vertical spacing between images based on a preset image overlap ratio, and determining a target cutting spacing corresponding to the target reference axis from the horizontal spacing between images or the vertical spacing between images;

[0016] The point cloud data is cut along the target reference axis according to the target cutting distance to obtain a plurality of target point cloud sections.

[0017] Optionally, the step of intercepting a point cloud ring corresponding to each target point cloud section from the point cloud data includes:

[0018] For each target point cloud section, the position of the target point cloud section is taken as the center, and the point clouds on both sides of the target point cloud section are intercepted from the point cloud data according to a preset safety distance along the positive and negative directions of the target reference axis to obtain the point cloud annulus corresponding to the target point cloud section.

[0019] Optionally, the collision sphere includes a first sphere and a second sphere having the same center, the radius of the first sphere is the photographing distance, and the radius of the second sphere is the safety distance;

[0020] The center of the collision sphere moves within the plane where the target point cloud section is located.

[0021] Optionally, the step of controlling a pre-constructed collision sphere to move around the target point cloud section to determine an envelope corresponding to the target point cloud section according to collision conditions between the collision sphere and the target point cloud section and the point cloud annulus during the movement includes:

[0022] Construct a collision sphere based on the preset camera distance and safety distance;

[0023] Control the collision sphere to move around the target point cloud section, and adjust the forward angle of the collision sphere according to the collision situation between the collision sphere and the target point cloud section and the point cloud annulus during the movement until the envelope line corresponding to the target point cloud section is obtained; wherein the distances between the points contained in the envelope line and the target point cloud section and the point cloud annulus are both greater than or equal to the safety distance, and the distances between the points contained in the envelope line and the target point cloud section are the minimum value.

[0024] Optionally, the step of controlling the collision sphere to move around the target point cloud section, and adjusting the forward angle of the collision sphere according to the collision conditions between the collision sphere and the target point cloud section and the point cloud annulus during the movement until an envelope corresponding to the target point cloud section is obtained includes:

[0025] According to a preset moving step, the collision sphere is controlled to move toward the target point cloud section;

[0026] During the movement, if the collision sphere collides with the target point cloud section and / or the point cloud annulus, adjusting the forward angle of the collision sphere;

[0027] Continue to control the collision sphere to move in a direction close to the target point cloud section according to the movement step size and the forward angle until the distance between the current position of the collision sphere and the position of the first collision point is less than a preset threshold, and then control the collision sphere to stop moving;

[0028] An envelope corresponding to the target point cloud section is generated based on the position of each collision point.

[0029] Optionally, if the collision sphere collides with the target point cloud section and / or the point cloud annulus, the step of adjusting the forward angle of the collision sphere includes:

[0030] If the first sphere in the collision sphere collides with the target point cloud section, or the second sphere in the collision sphere collides with the point cloud ring, the collision point position is recorded, the collision sphere is controlled to move once in the direction away from the target point cloud section according to the movement step, and the forward angle of the collision sphere is adjusted.

[0031] Optionally, the method further includes:

[0032] determining a detection range according to the current position of the collision sphere;

[0033] Determining whether the number of times the collision sphere moves within the detection range is greater than a preset number threshold;

[0034] If yes, the moving step is increased at least once to continue controlling the movement of the collision sphere according to the increased moving step until the current position of the collision sphere is outside the detection range, and the moving step is restored.

[0035] Optionally, the step of generating an image acquisition route corresponding to the region of interest based on the points included in the envelope includes:

[0036] Determine a horizontal spacing between images and a vertical spacing between images based on a preset image overlap ratio, and use the other of the horizontal spacing between images and the vertical spacing between images, excluding the target cutting spacing corresponding to the target reference axis, as a target extraction spacing;

[0037] Extracting a waypoint from the envelope according to the target extraction interval, determining the nearest point corresponding to the waypoint from the target point cloud section, and obtaining an image acquisition angle at the waypoint based on a vector between the waypoint and the nearest point;

[0038] An image acquisition route corresponding to the area of ​​interest is generated based on the waypoints and the image acquisition angles at the waypoints.

[0039] Optionally, after the step of generating the image acquisition route corresponding to the region of interest based on the points included in the envelope, the method further includes:

[0040] For each waypoint in the image acquisition route, determining whether the image acquisition angle at the waypoint and the angle difference between the image acquisition angles at adjacent waypoints corresponding to the waypoint are greater than a preset difference threshold;

[0041] If yes, performing angle interpolation between the waypoint and the adjacent waypoints to obtain a plurality of interpolated waypoints and image acquisition angles at the interpolated waypoints;

[0042] The interpolation waypoints and the image acquisition angles at the interpolation waypoints are added to the image acquisition route to obtain a target image acquisition route.

[0043] In a second aspect, an embodiment of the present application further provides a three-dimensional modeling route planning device, comprising:

[0044] a point cloud acquisition module configured to acquire point cloud data corresponding to a region of interest;

[0045] a section and annulus determination module configured to cut the point cloud data along a target reference axis based on a preset image overlap ratio to obtain a plurality of target point cloud sections, and to extract a point cloud annulus corresponding to each target point cloud section from the point cloud data;

[0046] an envelope determination module configured to control a pre-constructed collision sphere to move around the target point cloud section, so as to determine the envelope corresponding to the target point cloud section according to collision conditions between the collision sphere and the target point cloud section and the point cloud annulus during the movement;

[0047] The route generation module is configured to generate an image acquisition route corresponding to the area of ​​interest based on the points included in the envelope.

[0048] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0049] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0050] The embodiments of the present application provide a three-dimensional modeling route planning method, device, electronic device and storage medium, which first obtain point cloud data corresponding to the area of ​​interest; then, based on a preset image overlap rate, the point cloud data is cut along the target reference axis to obtain multiple target point cloud sections, and the point cloud annulus corresponding to each target point cloud section is intercepted from the point cloud data; then, a pre-constructed collision sphere is controlled to move around the target point cloud section to determine the envelope corresponding to the target point cloud section according to the collision situation between the collision sphere and the target point cloud section and the point cloud annulus during the movement; finally, an image acquisition route corresponding to the area of ​​interest is generated based on the points contained in the envelope. The above method cuts the point cloud data based on the image overlap rate to obtain multiple target point cloud sections and their corresponding point cloud rings, takes the target point cloud section and its corresponding point cloud ring as the minimum processing unit, and generates a corresponding envelope line according to the collision situation between the collision sphere and the minimum processing unit. The points within the envelope line are the points with the smallest distance from the target point cloud section on the basis of satisfying the condition of no collision, and then generates an image acquisition route based on the points contained in the envelope line. The embodiment of the present application can significantly improve the accuracy and automation of the planned route, avoid collision accidents of drones, thereby helping to improve the accuracy of subsequent three-dimensional modeling and reduce safety hazards during drone navigation.

[0051] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description or be understood by practicing the present application. The objectives and other advantages of the present application are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0052] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] FIG1 is a flow chart of a three-dimensional modeling route planning method provided in an embodiment of the present application;

[0055] FIG2 is a schematic diagram of an envelope provided in an embodiment of the present application;

[0056] FIG3 is a top view of an envelope provided in an embodiment of the present application;

[0057] FIG4 is a schematic diagram of an image acquisition route provided in an embodiment of the present application;

[0058] FIG5 is a schematic structural diagram of a three-dimensional modeling route planning device provided in an embodiment of the present application;

[0059] FIG6 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0061] Currently, related technologies use a rough 3D frame model of a building to generate a drone flight path. This results in low drone flight path accuracy and a high probability of collision between the drone and the building. This not only affects the accuracy of subsequent 3D modeling, but also creates certain safety hazards during the drone's image acquisition process. Based on this, the present application provides a 3D modeling route planning method, device, electronic device, and storage medium that can significantly improve the accuracy and automation of planned routes, avoid drone collision accidents, and thus help improve the accuracy of subsequent 3D modeling and reduce safety hazards during drone navigation.

[0062] To facilitate understanding of this embodiment, a three-dimensional modeling route planning method disclosed in an embodiment of the present application is first described in detail. Referring to the flowchart of a three-dimensional modeling route planning method shown in FIG1 , the method mainly includes the following steps S102 to S108:

[0063] Step S102: Acquire point cloud data corresponding to the region of interest.

[0064] The region of interest may also be referred to as a scanned scene or a scanned object.

[0065] In step S104 , based on a preset image overlap ratio, the point cloud data is cut along the target reference axis to obtain a plurality of target point cloud sections, and a point cloud ring corresponding to each target point cloud section is intercepted from the point cloud data.

[0066] Among them, the target reference axis can be the X-axis, Y-axis, and Z-axis in the coordinate system corresponding to the area of ​​interest, the target point cloud section can be understood as a section composed of the point cloud of the outer contour of the area of ​​interest, and the point cloud annulus can be understood as a point cloud with a certain thickness intercepted from both sides of the target point cloud section. The interception thickness is related to the preset safety distance. The target point cloud section and the point cloud annulus are combined as the minimum processing unit.

[0067] In one embodiment, the X-axis, Y-axis, and Z-axis are selected in sequence as target reference axes; the horizontal spacing between images and the vertical spacing between images are determined according to the image overlap rate, and the target cutting spacing corresponding to the target reference axis is selected from the two spacings; the point cloud data is cut along the target reference axis according to the target cutting spacing to obtain a target point cloud section; point clouds with a certain thickness intercepted from both sides of the target point cloud section are used as the point cloud annulus corresponding to the target point cloud section, and the target point cloud section and its corresponding point cloud annulus are used as the minimum processing units.

[0068] Step S106 , controlling the pre-built collision sphere to move around the target point cloud section, so as to determine the envelope corresponding to the target point cloud section according to the collision conditions between the collision sphere and the target point cloud section and the point cloud ring during the movement.

[0069] Among them, the distances between the points contained in the envelope and the target point cloud section and the point cloud ring are greater than or equal to the safety distance, and the distance between the points contained in the envelope and the target point cloud section is the minimum value; the collision sphere includes a first sphere and a second sphere with the same center. The radius of the first sphere is the shooting distance, and the radius of the second sphere is the safety distance.

[0070] In one embodiment, the collision sphere can be controlled to move toward the minimum processing unit according to a preset movement step size. When a collision occurs, the collision sphere is controlled to retreat a distance of the movement step size. The forward angle of the collision sphere is then adjusted, and the collision sphere is controlled to continue moving toward the minimum processing unit. This process is repeated to achieve the purpose of moving the collision sphere around the minimum processing unit. The position of each collision point during the movement is recorded. The collision point position is also the position of the collision sphere's center at the time of collision. The envelope corresponding to the minimum processing unit is obtained by connecting the positions of each collision point.

[0071] Step S108 : generating an image acquisition route corresponding to the region of interest based on the points included in the envelope.

[0072] In one embodiment, waypoints can be extracted from the envelope according to the horizontal and vertical spacing between images to generate an initial image acquisition route. Furthermore, each waypoint within the image acquisition route can be traversed to check whether the image acquisition angle between two adjacent waypoints has a sudden change. If so, the image acquisition angle between the two waypoints needs to be interpolated to generate a new waypoint. The newly generated waypoint and its corresponding image acquisition angle are then added to the aforementioned image acquisition route to obtain the target image acquisition route.

[0073] The three-dimensional modeling route planning method provided in the embodiment of the present application cuts the point cloud data based on the image overlap rate to obtain multiple target point cloud sections and their corresponding point cloud rings, takes the target point cloud section and its corresponding point cloud ring as the minimum processing unit, and generates a corresponding envelope line according to the collision situation between the collision sphere and the minimum processing unit. The points within the envelope line are the points with the smallest distance from the target point cloud section on the basis of satisfying the condition of no collision, and then generate an image acquisition route based on the points contained in the envelope line. The embodiment of the present application can significantly improve the accuracy and automation of the planned route, avoid collision accidents of drones, thereby helping to improve the accuracy of subsequent three-dimensional modeling and reduce safety hazards during drone navigation.

[0074] The embodiment of the present application aims to automatically generate a fine scanning route for a region of interest (referred to as a scanned scene or scanned object) given a given input. The data (photos) collected by executing this route can ensure: (1) full coverage of the visible surface of the scanned object; (2) a fixed distance between all photos and the surface of the scanned object; and (3) a sufficient overlap rate between adjacent photos. Therefore, using this data for three-dimensional reconstruction can obtain a fine three-dimensional model of the scanned object. In addition, the embodiment of the present application can also ensure that the generated route is safe and does not collide with the region of interest.

[0075] First, some descriptive terms are defined. (1) 3D modeling: Using 3D reconstruction software, with a series of photos and photo poses as input, a 3D digital model of the area of ​​interest is obtained through steps such as feature point extraction and matching, motion structure recovery, multi-view reconstruction, triangulation, and mapping. (2) Fineness: The fineness of the 3D model is reflected in the accuracy of the volume and structure and the fineness of the mapping, including whether the small-scale structure is accurately reconstructed and the surface sampling rate of the mapping (the actual size of the unit pixel on the model surface). (3) Overlap ratio: The ratio of the vertical and horizontal overlapping parts of two adjacent photos to the width and height of the frame, respectively.

[0076] On this basis, the embodiment of the present application provides a specific implementation method of a three-dimensional modeling route planning method.

[0077] After executing step S102, it can be determined whether there are any missing point cloud data (such as the point cloud below the ground in a plane scene is missing). If there are any missing points, three straight edges can be automatically generated to complete the cross-section geometry into a trapezoidal or rectangular shape to ensure that the collision sphere is successfully surrounded.

[0078] Regarding the aforementioned step S104, the embodiment of the present application provides an implementation method for cutting the point cloud data along the target reference axis based on a preset image overlap ratio to obtain multiple target point cloud sections, as shown in the following steps A1 to A3:

[0079] Step A1: Determine at least one target reference axis from the coordinate axes included in the coordinate system corresponding to the region of interest.

[0080] In one embodiment, the present application can scan the region of interest in three dimensions: x, y, and z. The x, y, and z axes refer to the x, y, and z axes of the object's coordinate system. The object's coordinate system typically has its z axis parallel to the world coordinate system and undergoes a rotational and translational transformation with the world coordinate system. In practical applications, the z-axis is often used for scanning cylindrical objects and vertical surfaces, while the x and y-axis dimensions are often used for scanning the tops of cylindrical objects and vertical surfaces, as well as for scanning planar surfaces.

[0081] Optionally, at the initial stage, scenes are categorized based on human judgment: 1. Columnar without a top; 2. Columnar with a top; 3. Vertical without a top; 4. Vertical with a top; 5. Planar. Columnar and vertical scenes use the z-axis, while vertical and planar scenes use the xy-axis. When axial routes exist, coverage of the z-axis is prioritized over coverage of the xy-axis. The amount of overlap between the xy-axis and z-axis coverage can be manually configured.

[0082] For example, the embodiments of the present application are described with respect to the z-axis dimension, that is, the z-axis is used as the target reference axis, and there is no essential difference in the algorithm between the x-axis, y-axis dimensions and the z-axis dimension.

[0083] Step A2: determining the horizontal spacing and the vertical spacing between images based on a preset image overlap ratio, and determining the target cutting spacing corresponding to the target reference axis from the horizontal spacing or the vertical spacing between images.

[0084] The image overlap ratio includes the horizontal overlap ratio and the vertical overlap ratio. In one embodiment, the spacing between two adjacent horizontal and vertical photos, i.e., the horizontal spacing h or the vertical spacing v between images, can be calculated based on the horizontal and vertical overlap ratios set by the user. For example, when the z-axis is the target reference axis, the vertical spacing v between images will be used as the target cutting spacing, i.e., the spacing for cutting point cloud data, and the horizontal spacing h between images will be used as the target extraction spacing, i.e., the spacing between points within the extraction envelope.

[0085] Step A3: Cut the point cloud data along the target reference axis according to the target cutting distance to obtain multiple target point cloud sections.

[0086] For example, assuming the z-axis is the target reference axis, the point cloud data is sliced ​​at equal intervals along the z-axis to obtain a series of initial point cloud slices. The spacing between two adjacent initial point cloud slices is the vertical spacing v between images. The initial point cloud slices have a certain thickness, which is an empirical parameter related to accuracy. The target point cloud slices are obtained by slicing the initial point cloud slices to a thickness of 0.

[0087] In a specific embodiment, multiple cutting points can be determined based on the vertical spacing v between the above-mentioned images; for each cutting point, the point cloud data is cross-sectioned along the z-axis with the cutting point as the center according to a preset thickness, and an initial point cloud section with equal spacing and a certain thickness can be obtained; the initial point cloud section is projected into two-dimensional space to obtain a target point cloud section with a thickness of 0.

[0088] For the aforementioned step S104, the embodiment of the present application also provides an implementation method for cutting out the point cloud ring corresponding to each target point cloud section from the point cloud data. Specifically: for each target point cloud section, the position of the target point cloud section is taken as the center, and the point clouds on both sides of the target point cloud section are cut out from the point cloud data along the positive and negative directions of the target reference axis according to a preset safety distance to obtain the point cloud ring corresponding to the target point cloud section.

[0089] For example, the user-set shooting distance d is obtained. O and safety distance d S ; Take the target point cloud section as the center and cut a section with a thickness of d along the positive direction of the z axis S The point cloud, and the thickness of d along the negative direction of the z axis S The point cloud with a thickness of 2d S Point cloud ring.

[0090] The target point cloud section and the point cloud ring are combined to form the minimum processing unit for subsequent processing, and the point cloud within the minimum processing unit becomes the collision point cloud.

[0091] For the aforementioned step S106, the embodiment of the present application provides an implementation method for constructing a collision sphere based on a preset shooting distance and a safety distance. Specifically, for each minimum processing unit, a concentric first sphere and a second sphere are defined, and the radius of the first sphere is d. Q , the radius of the second sphere is d S , the first sphere and the second sphere are combined to form a collision sphere. In the specific implementation, the center of the collision sphere can only move within the plane where the target point cloud section is located.

[0092] In one embodiment, the initial position of the center of the collision sphere is selected from the rectangular bounding box of the collision point cloud and then expanded outward by max(d O ,d S ), ensure that the collision sphere and the collision point cloud have no contact at the beginning.

[0093] On this basis, the embodiment of the present application also provides an implementation method for controlling a pre-constructed collision sphere to move around the target point cloud section, so as to determine the envelope line corresponding to the target point cloud section according to the collision situation between the collision sphere and the target point cloud section and the point cloud ring during the movement. The collision sphere can be constructed based on the preset shooting distance and safety distance; then the collision sphere is controlled to move around the target point cloud section, and during the movement, the forward angle of the collision sphere is adjusted according to the collision situation between the collision sphere and the target point cloud section and the point cloud ring until the envelope line corresponding to the target point cloud section is obtained.

[0094] In a specific embodiment, see the following steps B1 to B4:

[0095] Step B1: Control the collision sphere to move toward the target point cloud section according to a preset movement step size.

[0096] In one embodiment, the collision sphere is moved in a straight line toward the collision point cloud. The movement is discontinuous, each movement is in steps, and the length of each step (ie, movement compensation) is a parameter related to accuracy.

[0097] Step B2: During the movement process, if the collision sphere collides with the target point cloud section and / or point cloud annulus, the forward angle of the collision sphere is adjusted.

[0098] Specifically, if the first sphere in the collision sphere collides with the target point cloud section, or the second sphere in the collision sphere collides with the point cloud ring, the collision point position is recorded, the collision sphere is controlled to move once in the direction away from the target point cloud section according to the moving step size, and the forward angle of the collision sphere is adjusted.

[0099] In one embodiment, when the radius is d OWhen the first sphere collides with the target point cloud section, the collision sphere stops moving, or when the radius is d S When the second sphere collides with the point cloud ring, the collision sphere stops moving. After the collision stops, it moves in the opposite direction by a distance equal to the movement step, rotates the forward angle counterclockwise by a certain angle (an empirical parameter related to accuracy), and then resumes its forward movement.

[0100] In step B3, the collision sphere is controlled to move toward the target point cloud slice according to the movement step size and forward angle until the distance between the current position of the collision sphere and the first collision point is less than a preset threshold, at which point the collision sphere is controlled to stop moving. The current position is the current location of the center of the collision sphere.

[0101] In one embodiment, this cycle is repeated until the collision sphere rolls one circle on the collision point cloud surface. The criterion for determining whether the collision sphere has rolled one circle is that the center of the collision sphere returns to the vicinity of the position where the first collision occurred.

[0102] Step B4: Generate an envelope corresponding to the target point cloud section based on the position of each collision point.

[0103] In one embodiment, the position of the center of the ball at each step of the movement is recorded, and these position points are all located on an envelope line of the section. Referring to a schematic diagram of an envelope line shown in FIG2 , this envelope line has the following properties: for any point on the envelope line, the distance from the nearest point on the point cloud ring is d1, and d1 ≥ d S ; The distance from the nearest point on the target point cloud section is d2, and d2≥d S , and under the conditions of satisfying the above two constraints, d2 takes the possible minimum value.

[0104] Considering that the collision sphere may fall into a depression on the collision point cloud surface and be unable to get out during its movement, the embodiment of the present application can automatically detect whether the collision sphere is in this situation and try to jump out of the depression by increasing the movement step size until the collision sphere jumps out of the depression. Then, the collision sphere continues to move around the target point cloud section according to the original movement step size. In the specific implementation, please refer to the following steps C1 to C3:

[0105] Step C1, determining the detection range according to the current position of the collision sphere.

[0106] In one embodiment, the detection range can be obtained by taking the center of the collision sphere as the center of the circle and combining it with a preset radius.

[0107] Step C2: determining whether the number of times the collision sphere moves within the detection range is greater than a preset number threshold.

[0108] In one embodiment, if the number of times the collision sphere moves within the detection range is greater than a preset number threshold, but still leaves the detection range, it is considered that the collision sphere is trapped in the recess and cannot be moved out, and the following step C3 will be executed at this time; if the number of movements is less than the preset number threshold, the collision sphere can continue to be controlled to move around the target point cloud section.

[0109] In step C3, the moving step is increased at least once to continue controlling the movement of the collision sphere according to the increased moving step until the current position of the collision sphere is outside the detection range, and the moving step is restored.

[0110] In one embodiment, the moving step size can be adjusted once according to a preset increment, and the movement is advanced and retreated according to the moving step size, and it is detected whether the collision sphere can jump out of the depression; if not, the moving step size is continued to be adjusted once according to the preset increment until the collision sphere jumps out of the depression, that is, the current position of the collision sphere is outside the detection range. At this time, the adjusted moving step size is restored to the original value to continue to control the collision sphere to move around the target point cloud section according to the original value.

[0111] Regarding the aforementioned step S108, the embodiment of the present application further provides an implementation method for generating an image acquisition route corresponding to the region of interest based on the points contained in the envelope, as shown in the following steps D1 to D3:

[0112] In step D1 , the horizontal spacing and the vertical spacing between images are determined based on a preset image overlap ratio, and the other spacing between the horizontal spacing and the vertical spacing between images, excluding the target cutting spacing corresponding to the target reference axis, is used as the target extraction spacing.

[0113] Exemplarily, assuming that the z-axis is the target reference axis, the aforementioned embodiment uses the vertical spacing v between images as the target cutting spacing, and in this case uses the horizontal spacing h between images as the target extraction spacing.

[0114] Step D2: extract waypoints from the envelope according to the target extraction interval, determine the nearest point corresponding to the waypoint from the target point cloud section, and obtain the image acquisition angle at the waypoint based on the vector between the waypoint and the nearest point.

[0115] For example, see Figure 3, a top view of the envelope, which illustrates the image acquisition angle (also referred to as the shooting direction vector) for each point within the envelope. In one embodiment, points can be extracted from the envelope according to the horizontal spacing h between images. These points are the positions of the drone at the time of the photo, i.e., waypoints. For each waypoint, the point closest to the waypoint is determined from the target point cloud section, referred to as the closest point. The image acquisition angle is the vector between the waypoint and the closest point.

[0116] Step D3: generating an image acquisition route corresponding to the area of ​​interest based on the waypoints and the image acquisition angles at the waypoints.

[0117] In one embodiment, these waypoints are connected in the order in which they were acquired on the envelope to obtain an executable drone route. The routes of each minimum processing unit are combined to obtain a fine scanning route in the z-axis direction of the entire area of ​​interest, also known as the image acquisition route.

[0118] Furthermore, scanning the top of a cylindrical object or a flat scene requires simultaneous x- and y-axis scanning to ensure complete coverage of the object from all directions. The algorithm flow is identical to that for the z-axis. Figure 4 shows a schematic diagram of an image acquisition route. By combining the x-, y-, and z-axis image acquisition routes, an image acquisition route for the entire region of interest can be generated.

[0119] Furthermore, in the case where the overlap rate decreases due to a sudden change in the image acquisition angle, the embodiment of the present application will automatically detect whether this situation exists and perform angle interpolation between the two sudden image acquisition angles, adding new waypoints to ensure the overlap rate. Specifically, see the following steps E1 to E3:

[0120] Step E1: for each waypoint in the image acquisition route, determine whether the image acquisition angle at the waypoint and the angle difference between the image acquisition angle at the adjacent waypoint corresponding to the waypoint are greater than a preset difference threshold.

[0121] Exemplarily, the angle difference between the image acquisition angle at the current waypoint and the image acquisition angle at the next waypoint is calculated, and when the angle difference is greater than a preset difference threshold, the following step E2 is executed; when the angle difference is less than the preset difference threshold, the next waypoint is continued to be traversed until the angle difference between two adjacent waypoints in the image acquisition route is less than the preset difference threshold.

[0122] Step E2: If yes, perform angle interpolation between the waypoint and adjacent waypoints to obtain multiple interpolation waypoints and image acquisition angles at the interpolation waypoints.

[0123] Exemplarily, the number of interpolation waypoints may be preset or determined according to the angle difference, and a plurality of interpolation waypoints and their corresponding image acquisition angles may be determined according to the number.

[0124] Step E3: Add the interpolated waypoints and the image acquisition angles at the interpolated waypoints to the image acquisition route to obtain the target image acquisition route.

[0125] In one embodiment, the waypoint planning termination condition for the z-axis is the completion of processing all "cells" distributed from one end to the other along the z-axis of the scanned object point cloud. Similarly, the x-axis and y-axis termination conditions are the same as those for the z-axis.

[0126] In summary, the embodiments of the present application provide a method for generating a route for refined 3D modeling. This method allows a drone to maintain a fixed distance from the scanned object while ensuring that the route does not collide with the scanned object, and captures the entire visible surface of the scanned object while maintaining a guaranteed overlap rate.

[0127] Based on the above embodiments, the present invention provides a three-dimensional modeling route planning device. Referring to FIG5 , a schematic diagram of a three-dimensional modeling route planning device is shown. The device mainly includes the following parts:

[0128] a point cloud acquisition module 502 configured to acquire point cloud data corresponding to the region of interest;

[0129] The section and annulus determination module 504 is configured to cut the point cloud data along the target reference axis based on a preset image overlap ratio to obtain a plurality of target point cloud sections, and to extract a point cloud annulus corresponding to each target point cloud section from the point cloud data;

[0130] An envelope determination module 506 is configured to control a pre-constructed collision sphere to move around the target point cloud section, so as to determine an envelope corresponding to the target point cloud section according to collision conditions between the collision sphere, the target point cloud section, and the point cloud annulus during the movement;

[0131] The route generation module 508 is configured to generate an image acquisition route corresponding to the region of interest based on the points included in the envelope.

[0132] The three-dimensional modeling route planning device provided in the embodiment of the present application cuts the point cloud data based on the image overlap rate to obtain multiple target point cloud sections and their corresponding point cloud rings, takes the target point cloud section and its corresponding point cloud ring as the minimum processing unit, constructs a collision sphere in combination with the shooting distance and the safety distance, and generates a corresponding envelope line according to the collision situation between the collision sphere and the minimum processing unit. The points within the envelope line are the points with the smallest distance from the target point cloud section on the basis of satisfying the condition of no collision, and then generate an image acquisition route based on the points contained in the envelope line. The embodiment of the present application can significantly improve the accuracy and automation of the planned route, avoid collision accidents of drones, thereby helping to improve the accuracy of subsequent three-dimensional modeling and reduce safety hazards during drone navigation.

[0133] In one embodiment, the section and annulus determination module 504 is further configured to:

[0134] Determining at least one target reference axis from among the coordinate axes contained in the coordinate system corresponding to the region of interest;

[0135] Determine the horizontal spacing between images and the vertical spacing between images based on a preset image overlap ratio, and determine the target cutting spacing corresponding to the target reference axis from the horizontal spacing between images or the vertical spacing between images;

[0136] According to the target cutting distance, the point cloud data is cut along the target reference axis to obtain multiple target point cloud sections.

[0137] In one embodiment, the section and annulus determination module 504 is further configured to:

[0138] For each target point cloud section, the position of the target point cloud section is taken as the center, and the point clouds on both sides of the target point cloud section are intercepted from the point cloud data according to the preset safety distance along the positive and negative directions of the target reference axis to obtain the point cloud ring corresponding to the target point cloud section.

[0139] In one embodiment, the collision sphere includes a first sphere and a second sphere having the same center, the radius of the first sphere is the photographing distance, and the radius of the second sphere is the safety distance;

[0140] The center of the collision sphere moves within the plane where the target point cloud section is located.

[0141] In one embodiment, the envelope determination module 506 is further configured to:

[0142] Construct a collision sphere based on the preset camera distance and safety distance;

[0143] Control the collision sphere to move around the target point cloud section, and adjust the forward angle of the collision sphere according to the collision situation between the collision sphere and the target point cloud section and the point cloud annulus during the movement until the envelope line corresponding to the target point cloud section is obtained; wherein the distances between the points contained in the envelope line and the target point cloud section and the point cloud annulus are both greater than or equal to the safety distance, and the distances between the points contained in the envelope line and the target point cloud section are the minimum value.

[0144] In one embodiment, the envelope determination module 506 is further configured to:

[0145] According to the preset moving step, control the collision sphere to move towards the target point cloud section;

[0146] During the movement, if the collision sphere collides with the target point cloud section and / or the point cloud annulus, adjusting the forward angle of the collision sphere;

[0147] Continue to control the collision sphere to move toward the target point cloud slice according to the moving step size and forward angle until the distance between the current position of the collision sphere and the position of the first collision point is less than the preset threshold, and then control the collision sphere to stop moving;

[0148] Generate the envelope corresponding to the target point cloud section based on the position of each collision point.

[0149] In one embodiment, the envelope determination module 506 is further configured to:

[0150] If the first sphere in the collision sphere collides with the target point cloud section, or the second sphere in the collision sphere collides with the point cloud ring, the collision point position is recorded, the collision sphere is controlled to move once away from the target point cloud section according to the movement step, and the forward angle of the collision sphere is adjusted.

[0151] In one embodiment, the envelope determination module 506 is further configured to:

[0152] Determine the detection range based on the current position of the collision sphere;

[0153] Determine whether the number of times the collision sphere moves within the detection range is greater than a preset number threshold;

[0154] If so, the moving step is increased at least once to continue controlling the movement of the collision sphere according to the increased moving step until the current position of the collision sphere is outside the detection range, and the moving step is restored.

[0155] In one embodiment, the route generation module 508 is further configured to:

[0156] Determine the horizontal spacing between images and the vertical spacing between images based on a preset image overlap ratio, and use the other spacing between the horizontal spacing between images and the vertical spacing between images, excluding the target cutting spacing corresponding to the target reference axis, as the target extraction spacing;

[0157] Extract waypoints from the envelope according to the target extraction interval, determine the nearest point corresponding to the waypoint from the target point cloud section, and obtain the image acquisition angle at the waypoint based on the vector between the waypoint and the nearest point;

[0158] Based on the waypoints and the image acquisition angles at the waypoints, an image acquisition route corresponding to the area of ​​interest is generated.

[0159] In one embodiment, the route generation module 508 is further configured to:

[0160] For each waypoint in the image acquisition route, determine whether the image acquisition angle at the waypoint and the angle difference between the image acquisition angles at the adjacent waypoints corresponding to the waypoint are greater than a preset difference threshold;

[0161] If yes, then interpolate the angles between the waypoint and the adjacent waypoints to obtain multiple interpolated waypoints and image acquisition angles at the interpolated waypoints;

[0162] The interpolated waypoints and the image acquisition angles at the interpolated waypoints are added to the image acquisition route to obtain the target image acquisition route.

[0163] The device provided in the embodiment of the present application has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.

[0164] An embodiment of the present application provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.

[0165] Figure 6 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63. The processor 60, the communication interface 63 and the memory 61 are connected via the bus 62; the processor 60 is configured to execute an executable module stored in the memory 61, such as a computer program.

[0166] The memory 61 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element communicates with at least one other network element via at least one communication interface 63 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0167] The bus 62 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG6 shows only one bidirectional arrow, but this does not mean that there is only one bus or one type of bus.

[0168] Among them, the memory 61 is configured to store a program, and the processor 60 executes the program after receiving an execution instruction. The method executed by the device for flow process definition disclosed in any of the aforementioned embodiments of the present application can be applied to the processor 60 or implemented by the processor 60.

[0169] The processor 60 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 60 or software instructions. The above processor 60 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 61, and processor 60 reads the information in memory 61 and, in conjunction with its hardware, completes the steps of the above method.

[0170] The computer program product of the readable storage medium provided in the embodiments of the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the previous method embodiments and will not be repeated here.

[0171] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0172] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims. Industrial Applicability

[0173] In summary, the present application provides a three-dimensional modeling route planning method, device, electronic device and storage medium, which can significantly improve the accuracy and automation procedures of planned routes, avoid collision accidents of drones, thereby helping to improve the accuracy of subsequent three-dimensional modeling and reduce safety hazards during drone navigation.

Claims

1. A three-dimensional modeling route planning method, characterized in that: include: Get the point cloud data corresponding to the area of ​​interest; Based on a preset image overlap rate, the point cloud data is cut along a target reference axis to obtain a plurality of target point cloud sections, and a point cloud ring corresponding to each target point cloud section is intercepted from the point cloud data; Controlling a pre-constructed collision sphere to move around the target point cloud section, so as to determine an envelope corresponding to the target point cloud section according to a collision condition between the collision sphere and the target point cloud section and the point cloud annulus during the movement; An image acquisition route corresponding to the region of interest is generated based on the points included in the envelope.

2. The route planning method of three-dimensional modeling according to claim 1, characterized in that: The step of cutting the point cloud data along the target reference axis to obtain a plurality of target point cloud sections based on a preset image overlap rate comprises: Determining at least one target reference axis from the coordinate axes included in the coordinate system corresponding to the region of interest; Determine a horizontal spacing between images and a vertical spacing between images based on a preset image overlap rate, and determine a target cutting spacing corresponding to the target reference axis from the horizontal spacing between images or the vertical spacing between images; The point cloud data is cut along the target reference axis according to the target cutting distance to obtain a plurality of target point cloud sections.

3. The route planning method of three-dimensional modeling according to claim 2, characterized in that: The step of extracting a point cloud ring corresponding to each target point cloud section from the point cloud data comprises: For each of the target point cloud sections, the position of the target point cloud section is taken as the center, and the point clouds on both sides of the target point cloud section are intercepted from the point cloud data along the positive and negative directions of the target reference axis according to a preset safety distance to obtain the point cloud annulus corresponding to the target point cloud section.

4. The route planning method of three-dimensional modeling according to any one of claims 1 to 3, characterized in that: The collision sphere includes a first sphere and a second sphere with the same center, the radius of the first sphere is the shooting distance, and the radius of the second sphere is the safety distance; The center of the collision sphere moves within the plane where the target point cloud section is located.

5. The route planning method of three-dimensional modeling according to any one of claims 1 to 4, characterized in that: The step of controlling a pre-constructed collision sphere to move around the target point cloud section to determine an envelope corresponding to the target point cloud section according to a collision condition between the collision sphere and the target point cloud section and the point cloud annulus during the movement includes: Construct a collision sphere based on the preset camera distance and safety distance; The collision sphere is controlled to move around the target point cloud section, and during the movement, the forward angle of the collision sphere is adjusted according to the collision conditions between the collision sphere and the target point cloud section and the point cloud annulus, until an envelope corresponding to the target point cloud section is obtained; wherein the distances from the points contained in the envelope to the target point cloud section and the point cloud annulus are both greater than or equal to the safety distance, and the distance from the points contained in the envelope to the target point cloud section is a minimum value.

6. The route planning method of three-dimensional modeling according to claim 5, characterized in that: The step of controlling the collision sphere to move around the target point cloud section, and adjusting the forward angle of the collision sphere according to the collision conditions between the collision sphere and the target point cloud section and the point cloud annulus during the movement until an envelope corresponding to the target point cloud section is obtained includes: According to a preset moving step length, the collision sphere is controlled to move toward a direction close to the target point cloud section; During the movement, if the collision sphere collides with the target point cloud section and / or the point cloud annulus, adjusting the forward angle of the collision sphere; Continue to control the collision sphere to move in a direction close to the target point cloud section according to the moving step length and the advancing angle until the distance between the current position of the collision sphere and the position of the first collision point is less than a preset threshold, and control the collision sphere to stop moving; An envelope corresponding to the target point cloud section is generated based on the position of each collision point.

7. The route planning method of three-dimensional modeling according to claim 6, characterized in that: If the collision sphere collides with the target point cloud section and / or the point cloud annulus, the step of adjusting the forward angle of the collision sphere comprises: If the first sphere in the collision sphere collides with the target point cloud section, or the second sphere in the collision sphere collides with the point cloud ring, the collision point position is recorded, the collision sphere is controlled to move once in a direction away from the target point cloud section according to the moving step, and the forward angle of the collision sphere is adjusted.

8. The route planning method of three-dimensional modeling according to claim 6, characterized in that: The method further comprises: Determining a detection range according to the current position of the collision sphere; Determine whether the number of times the collision ball moves within the detection range is greater than a preset number threshold; If yes, the moving step length is increased at least once, so as to continue to control the movement of the collision sphere according to the increased moving step length, until the current position of the collision sphere is outside the detection range, and the moving step length is restored.

9. The route planning method of three-dimensional modeling according to any one of claims 1 to 8, characterized in that: The step of generating an image acquisition route corresponding to the region of interest based on the points contained in the envelope line comprises: Determine the horizontal spacing between images and the vertical spacing between images based on a preset image overlap rate, and use another spacing between the horizontal spacing between images and the vertical spacing between images, except the target cutting spacing corresponding to the target reference axis, as the target extraction spacing; Extracting a waypoint from the envelope according to the target extraction interval, determining the nearest point corresponding to the waypoint from the target point cloud section, and obtaining an image acquisition angle at the waypoint based on a vector between the waypoint and the nearest point; An image acquisition route corresponding to the area of ​​interest is generated based on the waypoints and the image acquisition angles at the waypoints.

10. The route planning method of three-dimensional modeling according to any one of claims 1 to 9, characterized in that: After the step of generating the image acquisition route corresponding to the region of interest based on the points included in the envelope, the method further includes: For each waypoint in the image acquisition route, determining whether the image acquisition angle at the waypoint and the angle difference between the image acquisition angles at adjacent waypoints corresponding to the waypoint are greater than a preset difference threshold; If yes, then interpolating the angle between the waypoint and the adjacent waypoints to obtain a plurality of interpolated waypoints and image acquisition angles at the interpolated waypoints; The interpolation waypoints and the image acquisition angles at the interpolation waypoints are added to the image acquisition route to obtain a target image acquisition route.

11. A three-dimensional modeling route planning device, characterized in that: include: a point cloud acquisition module configured to acquire point cloud data corresponding to the region of interest; A section and annulus determination module is configured to cut the point cloud data along a target reference axis based on a preset image overlap rate to obtain a plurality of target point cloud sections, and to extract a point cloud annulus corresponding to each target point cloud section from the point cloud data; An envelope determination module is configured to control a pre-constructed collision sphere to move around the target point cloud section, so as to determine the envelope corresponding to the target point cloud section according to the collision conditions between the collision sphere and the target point cloud section and the point cloud annulus during the movement; The route generation module is configured to generate an image acquisition route corresponding to the area of ​​interest based on the points included in the envelope.

12. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 10.

Images