Photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition

Abstract

The invention provides a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition, belongs to the technical field of image processing, and is used for improving the surveying efficiency of photovoltaic construction. The method comprises the following steps: when an unmanned aerial vehicle flies over a target area based on a preset path in BIM modeling, the unmanned aerial vehicle shoots the target area at at least two associated positions to obtain at least two images; the target area is an area containing a construction position point in BIM modeling; the unmanned aerial vehicle fuses the at least two images through a front network of the neural network model to obtain fused image features, the fused image features are analyzed through a main network of the neural network model to obtain a target image, the target image is one of the at least two images marked with a construction point, and the construction point is marked with the target image. The construction position point in the target image corresponds to the construction position point in BIM modeling; and the unmanned aerial vehicle performs spraying marking on the actual position, mapped to the ground, of the construction position point in the target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition. BACKGROUND

[0002] With the increasing demand for renewable energy worldwide, photovoltaic power stations have been widely used as a clean energy solution. However, during the construction of photovoltaic power stations, manual surveying as one of the key steps has the problems of low efficiency and insufficient positioning accuracy. This paper will introduce this background technology in detail.

[0003] In the early stage of construction of photovoltaic power stations, manual surveying is an essential link. This process usually includes topographic mapping, land assessment, environmental impact assessment and other aspects. However, due to the complex and variable terrain, poor working environment and shortage of professional personnel, manual surveying is often inefficient and has insufficient positioning accuracy. These problems not only prolong the project cycle, but also may cause resource waste and environmental damage.

[0004] Therefore, how to improve the surveying efficiency is the current research problem. SUMMARY

[0005] The embodiment of the present application provides a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition, which is used to improve the surveying efficiency of photovoltaic construction.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] In the first aspect, a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition is provided, the method is applied to an unmanned aerial vehicle, and the method comprises the following steps: when the unmanned aerial vehicle flies to the upper space of a target area based on a preset path in BIM modeling, the unmanned aerial vehicle photographs the target area at least two associated positions to obtain at least two images; the target area is a region containing a construction position point in the BIM modeling; the unmanned aerial vehicle fuses the at least two images through a front-end network of a neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, the target image is an image in which a construction point is labeled in the at least two images, and the construction position point in the target image corresponds to the construction position point in the BIM modeling; and the unmanned aerial vehicle sprays and marks the actual position of the construction position point in the target image on the ground.

[0008] Optionally, when the UAV flies to the target area based on the preset path in the BIM modeling, the UAV photographs the target area at the associated at least two positions to obtain at least two images, including: the UAV determines to fly to the i-th target position on the preset path based on the preset path, the i-th target position corresponds to the i-th construction position point in the BIM modeling, i is an integer taking 1 to M, and M is an integer greater than 1; the UAV determines at least one associated position adjacent to the i-th target position based on the i-th target position to obtain at least two positions, the at least two positions including the i-th target position and the at least one associated position; and the UAV photographs the target area at the at least two positions respectively to obtain at least two images.

[0009] Optionally, the space in the BIM modeling is gridded into a plurality of subspaces, one subspace being a cubic region in the space, and the UAV determining to fly to the i-th target position on the preset path based on the preset path includes: the UAV determines whether the position of the UAV itself is located in the i-th target subspace in the plurality of subspaces through positioning during flight based on the preset path, the i-th target subspace being a subspace including the i-th target position; if the position of the UAV itself is located in the i-th target subspace, it indicates that the UAV flies to the i-th target position on the preset path based on the preset path; and the UAV determines at least one associated position adjacent to the i-th target position based on the i-th target position to obtain at least two positions, including: the UAV determines at least one subspace adjacent to the i-th target subspace in the plurality of subspaces based on the i-th target subspace, for any one of the at least one subspace, if the UAV flies to any position in any one of the at least one subspace, it is considered that the UAV is located in a corresponding one of the at least one associated position; and the UAV photographs the target area at the at least two positions respectively to obtain at least two images, including: the UAV photographs the target area in the i-th target subspace and the at least one subspace respectively in a direction perpendicular to the target area to obtain at least two images.

[0010] Optionally, the at least two images include a first image and a second image, the first image and the second image both contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the ith construction position point, and the reference positioning mark is associated with the plurality of construction position points in the BIM modeling; the unmanned aerial vehicle fuses the at least two images through a front network of the neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: the unmanned aerial vehicle convolves the first image and the second image through a convolution layer of the neural network model to correspondingly obtain a first feature set and a second feature set; the unmanned aerial vehicle performs three-dimensional space fusion on the first feature set and the second feature set through the front network to obtain the fused image features; and the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain the target image, and the construction position point in the target image is determined by the main network based on the preset position relationship.

[0011] Optionally, the front network includes a first unit, a second unit, a third unit and a fourth unit connected in a two-dimensional structure, the connection in the two-dimensional structure means that the first unit and the second unit are connected with the third unit, and the third unit is connected with the fourth unit through a buffer; and the unmanned aerial vehicle performs three-dimensional space fusion on the first feature set and the second feature set through the front network to obtain the fused image features, including: the neural network model synchronously inputs the first feature set and the second feature set one by one into the first unit and the second unit; in response to the input, the neural network model synchronously inputs the input of the buffer and the first feature set or the input of the buffer and the second feature set into the fourth unit to obtain the fused image features output by the fourth unit.

[0012] Optionally, the main network is a full-size fully connected layer network, and the front network is constructed by copying four units connected in a two-dimensional structure in the full-size fully connected layer network.

[0013] Optionally, the at least two images include a first image, a second image and a third image, the first image, the second image and the third image all contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the ith construction position point, and the reference positioning mark is associated with the plurality of construction position points in the BIM modeling; the unmanned aerial vehicle fuses the at least two images through a front network of the neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: the unmanned aerial vehicle respectively convolves the first image, the second image and the third image through a convolution layer of the neural network model to correspondingly obtain a first feature set, a second feature set and a third feature set; the unmanned aerial vehicle fuses the first feature set, the second feature set and the third feature set in a three-dimensional space through the front network to obtain the fused image features; and the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain the target image, and the construction position points in the target image are determined by the main network based on the preset position relationship.

[0014] Optionally, the front network includes a first unit, a second unit, a third unit and a fourth unit connected in a three-dimensional structure, the first unit is connected with the second unit and the third unit respectively, the second unit is connected with the first unit and the third unit respectively, the third unit is connected with the first unit and the second unit respectively, and the fourth unit is connected with the first unit, the second unit and the third unit respectively, the unmanned aerial vehicle fuses the first feature set and the second feature set in a three-dimensional space through the front network to obtain the fused image features, including: the neural network model synchronously inputs the first feature set, the second feature set and the third feature set into the first unit, the second unit and the third unit one by one to obtain the fused image features output by the fourth unit.

[0015] Optionally, the main network is a full-size fully connected layer network, and the front network is constructed by copying four units connected in a three-dimensional structure in the full-size fully connected layer network.

[0016] Optionally, the unmanned aerial vehicle sprays and marks the actual positions of the construction position points in the target image on the ground, including: the unmanned aerial vehicle maps the construction position points in the target image to an actual viewfinder, and the positions of the construction position points in the target image in the actual viewfinder can change with the movement of the unmanned aerial vehicle; and the unmanned aerial vehicle sprays and marks the ground based on the positions of the construction position points in the target image in the actual viewfinder, and the positions of the ground sprayed and marked are the actual positions of the construction position points in the target image on the ground.

[0017] In a second aspect, a computer readable storage medium is provided, including: a computer program or instructions; when the computer program or instructions run on a computer, the computer executes the method of the first aspect.

[0018] In summary, the above method has the following technical effects:

[0019] Firstly, the unmanned aerial vehicle can fly to the target area containing the construction position point in the BIM modeling based on the preset path in the BIM modeling, so as to take pictures of the target area at the associated at least two positions, and obtain at least two images. Secondly, the unmanned aerial vehicle can fuse the at least two images through the front-end network of the neural network model to obtain the fused image features, so as to realize the feature fusion in the spatial dimension. Then, the unmanned aerial vehicle can analyze the fused image features through the main network of the neural network model, so as to obtain the target image with the accurately labeled construction position point. Finally, the unmanned aerial vehicle can spray and mark the actual position of the construction position point on the ground in the target image. Compared with the existing manual survey marking method, the above scheme not only has strong robustness, that is, the construction position point labeling is relatively accurate and reliable by realizing the feature fusion in the spatial dimension. The construction positioning error is controlled within ±3cm, and the construction efficiency is improved by 42% compared with the traditional method. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition is provided for the embodiments of the present application.

[0021] Figures 2-3 An application scenario diagram of a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition is provided for the embodiments of the present application.

[0022] Figure 4A A model structure diagram of a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition is provided for the embodiments of the present application.

[0023] Figure 5 A structural diagram of an electronic device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0025] The present application will present various aspects, embodiments or features around a system which can include a plurality of devices, components, modules, etc. It should be understood and appreciated that each system can include additional devices, components, modules, etc., and / or can not include all the devices, components, modules, etc. discussed in connection with the accompanying drawings. In addition, combinations of these solutions can also be used.

[0026] In addition, in the embodiments of the present application, the words "example", "for example", and the like are used to represent that an example, an example, or an illustration is given. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner.

[0027] In the embodiments of the present application, "of", "corresponding" and "corresponding" can be used interchangeably. It should be pointed out that when the distinction is not emphasized, the meaning expressed is matching. In addition, the " / " mentioned in the present application can be used to represent the relationship of "or". The network architecture and business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that as the network architecture evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0028] Exemplary, Figure 1 The flowchart of the photovoltaic array positioning construction method based on the path planning and image recognition of the unmanned aerial vehicle provided by the embodiments of the present application is shown. The method can be applied to the unmanned aerial vehicle.

[0029] As Figure 1 The method flow of the photovoltaic array positioning construction method based on the path planning and image recognition of the unmanned aerial vehicle is as follows:

[0030] S101, when the unmanned aerial vehicle flies to the target area in the preset path based on BIM modeling, the unmanned aerial vehicle takes pictures of the target area at least two positions associated, and obtains at least two images.

[0031] The target area is a region containing the construction position point in the BIM modeling.

[0032] For example, the UAV can determine to fly to an i-th target position on the preset path based on the preset path, the i-th target position corresponds to an i-th construction position point in the BIM modeling, i is an integer taking 1 to M, and M is an integer greater than 1. In other words, throughout the process, the UAV needs to take a photo for the M construction position points in the BIM modeling in this way, and the preset path can also be planned based on the M construction position points in the BIM modeling, so that the UAV can fly to the M positions corresponding to the M construction position points in the BIM modeling in turn. The UAV can determine the change of its own position in real time during the flight based on the preset path, such as the change of latitude, longitude and height coordinates. The i-th target position is a position vertically above the i-th construction position point in the BIM modeling. When the UAV determines that the actual latitude, longitude and height coordinates of itself are consistent with the latitude, longitude and height coordinates corresponding to the i-th construction position, it is considered that the UAV has arrived at the i-th target position on the preset path.

[0033] Based on the i-th target position, the UAV determines at least one associated position adjacent to the i-th target position, and obtains at least two positions including the i-th target position and at least one associated position. Considering the multi-path effect (positioning jump caused by metal support reflection), IMU cumulative error (attitude angle drift after long-time flight), and body shaking caused by strong wind disturbance, the actual position of the UAV may deviate from the i-th target position, so it is necessary to take a photo at one or more associated positions to eliminate the influence of such errors through post-processing of the image.

[0034] Specifically, the associated positions can be achieved by spatial gridding. For example, the space in the BIM modeling is gridded into a plurality of subspaces, and one subspace is a cubic region in the space, such as a subspace of x*y*z cubic space, x, y, and z represent length, width, and height, respectively, in centimeters, such as x=10, y=10, z=10, or x=20, y=20, z=20, and the specific value is not limited and can be selected according to the actual situation. At this time, the latitude, longitude, and height coordinates of the i-th target position are also located in the corresponding subspace, such as the i-th target subspace, that is, the i-th target subspace is the subspace including the i-th target position. In this way, during the flight based on the preset path, the position of the unmanned aerial vehicle itself can be determined by positioning whether the unmanned aerial vehicle itself is located in the i-th target subspace in the plurality of subspaces. If the position of the unmanned aerial vehicle itself is located in the i-th target subspace, it means that the unmanned aerial vehicle flies to the i-th target position on the preset path based on the preset path, or it can be considered that when the unmanned aerial vehicle determines that the actual latitude, longitude, and height coordinates change to the latitude, longitude, and height coordinates corresponding to the i-th construction position, it is determined that the unmanned aerial vehicle is also located in the i-th target subspace. On this basis, since the unmanned aerial vehicle is pre-configured with the BIM modeling, the unmanned aerial vehicle can also determine at least one subspace adjacent to the i-th target subspace in the plurality of subspaces based on the i-th target subspace in the BIM modeling. For any one of the at least one subspace, if the unmanned aerial vehicle flies to any position in any one of the at least one subspace, it is considered that the unmanned aerial vehicle is located in a corresponding one of the at least one associated position. In this way, the unmanned aerial vehicle can respectively take pictures of the target region in the i-th target subspace and the at least one subspace in the direction perpendicular to the target region, and obtain at least two images.

[0035] It should be understood that if the target region is not horizontal but inclined, the influence of the slope also needs to be considered during shooting. Since the modeling of the target region is also in the BIM modeling, if the target region is a region with a slope, the i-th target position is perpendicular to the slope, which can be understood as a region above the target region. At this time, the BIM modeling can also configure the shooting direction, and at this time, the unmanned aerial vehicle can rotate the camera to make the shooting direction consistent with the shooting direction in the BIM modeling.

[0036] The target region is also provided with a reference positioning mark, which can be a metal piece, such as a metal plate with a metal reflective feature, so that the neural network model can recognize it. The reference positioning mark can be a plurality of marks, which can be sparsely and uniformly distributed between the actual M construction position points. Each reference positioning mark satisfies a preset positional relationship with the corresponding plurality of construction position points, so that the neural network model can determine the position of the construction position point based on the position of the reference positioning mark in the image.

[0037] For example, as shown in an example, Figure 2 The positions of the construction position points are distributed in a matrix form, and one reference positioning mark is arranged between every four construction position points, so that the neural network model can determine the positions of the four construction position points through the reference positioning mark. In this way, the image captured by the unmanned aerial vehicle for positioning the four construction position points all contains the one reference positioning mark.

[0038] For the convenience of understanding, an example is introduced below.

[0039] As shown in an example, Figure 3 The ith target position is located in the subspace A, and the unmanned aerial vehicle first flies to the subspace A to capture the target region in a direction perpendicular to the target region, to obtain a first image. Then, the unmanned aerial vehicle flies to the subspace B adjacent to the subspace A to capture the target region in a direction perpendicular to the target region, to obtain a second image. Optionally, the unmanned aerial vehicle can further fly to the subspace C adjacent to the subspace A to capture the target region in a direction perpendicular to the target region, to obtain a third image. The first image and the second image both contain a reference positioning mark, and the reference positioning mark and the ith construction position point satisfy a preset positional relationship. Optionally, if there is a third image, the third image also contains the reference positioning mark.

[0040] S102, the unmanned aerial vehicle fuses at least two images through the front-end network of the neural network model to obtain fused image features, and analyzes the fused image features through the main network of the neural network model to obtain a target image.

[0041] The target image is an image in which at least one construction point is labeled in the at least two images, and the construction position point in the target image corresponds to the construction position point in the BIM modeling.

[0042] Case 1:

[0043] The at least two images include the first image and the second image.

[0044] The drone uses convolutional layers of a neural network model to convolve the first and second images, respectively, to obtain a first feature set and a second feature set. The neural network model can be an improvement on a deep neural network model (CNN), such as a CNN with an added pre-processor network. Since the first and second images are at the same resolution, such as 1080P, the convolutional layers of the neural network model can perform convolutions on both images at the same scale, resulting in first and second feature sets of the same scale; that is, the number and dimension of feature vectors in the first and second feature sets are the same. Based on this, the drone can use the pre-processor network to fuse the first and second feature sets in three-dimensional space to obtain the fused image features. For example, ... Figure 4A As shown, the arrows indicate the direction of feature transmission. The pre-network includes a first unit, a second unit, a third unit, and a fourth unit connected in a two-dimensional structure. This two-dimensional structure means that the first and second units are connected to the third unit, and the third and fourth units are connected via a buffer. A unit can also be understood as a neuron or neural unit; for example, the first unit is the first neuron, the second unit is the second neuron, and so on. The main network is a full-size fully connected layer network. The pre-network is constructed by replicating the four units in the full-size fully connected layer network that form a two-dimensional structure (or by adding a buffer to the replica). This simplifies the network design and improves model performance in a relatively lightweight way. It should be understood that a full-size fully connected layer network can achieve output results through a large-scale neural network, such as position determination in this application. However, since the pre-network only has a few neurons, it cannot achieve output results on a large scale, but it can be used to achieve the feature fusion function in this application.

[0045] Taking any one of the units—Unit 1, Unit 2, Unit 3, and Unit 4—as an example, its function expression is as follows:

[0046]

[0047] The definition and explanation of each element are as follows:

[0048] xi represents the input, i is the neuron number (i=1 for the first unit, i=2 for the second unit, etc.), ωi represents the weight of the i-th unit, b is the bias, σ is the nonlinear function, and a is the output.

[0049] On this basis, the neural network model can synchronously input the first feature set and the second feature set one by one into the first unit and the second unit, and the first unit and the second unit input the fused features into the buffer. In response to the input of the buffer, the neural network model synchronously inputs the input of the buffer and the first feature set or the input of the buffer and the second feature set into the fourth unit to obtain the fused image features output by the fourth unit. At this time, it can be seen that the first feature set or the second feature set can be asynchronously fused twice to realize three-dimensional feature fusion of space through two feature sets, that is, the fusion of the first unit and the second unit is the fusion of the first two dimensions, and the fusion of the fourth unit is the fusion of the third dimension.

[0050] Then, the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain a target image, and a construction position point in the target image is determined by the main network based on a preset position relationship.

[0051] Case 2:

[0052] The at least two images include a first image, a second image, and a third image.

[0053] The unmanned aerial vehicle can convolve the first image, the second image, and the third image through the convolution layer of the neural network model to correspondingly obtain a first feature set, a second feature set, and a third feature set; the convolution layer of the neural network model can convolve the first image, the second image, and the third image in the same scale to obtain the first feature set, the second feature set, and the third feature set in the same scale, that is, the number of feature vectors and the dimension of the first feature set and the second feature set are the same.

[0054] The unmanned aerial vehicle can fuse the first feature set, the second feature set, and the third feature set in three-dimensional space through the front-end network to obtain fused image features; and the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain a target image, and a construction position point in the target image is determined by the main network based on a preset position relationship. For example, as shown in FIG. 6, the first image, the second image, and the third image are fused in three-dimensional space through the front-end network to obtain fused image features, and the fused image features are analyzed through the main network of the neural network model to obtain a target image. Figure 4BAs shown, the pre-network includes a first unit, a second unit, a third unit and a fourth unit connected in a three-dimensional structure, that is, the first unit is connected with the second unit and the third unit respectively, the second unit is connected with the first unit and the third unit respectively, the third unit is connected with the first unit and the second unit respectively, and the fourth unit is connected with the first unit, the second unit and the third unit respectively. The arrow direction represents the transmission direction of the features. The first unit and the second unit transmit to each other, the first unit unidirectionally transmits to the fourth unit, and the others are similar and will not be repeated. Among them, the main network is a full-size fully connected layer network, and the pre-network is constructed by copying the four units connected in a three-dimensional structure in the full-size fully connected layer network. In this way, the design of the network can be simplified to improve the performance of the model in a relatively lightweight manner. In addition, considering the stability of the output, the fourth unit can also be configured with a buffer between the first unit, the second unit and the third unit, that is, three buffers. In the case that all three buffers have input, the neural network model triggers the respective input features of the three buffers to be synchronously input to the fourth unit. It should be understood that for the full-size fully connected layer network, it can realize the output of the result through a large-scale neuron network, such as the position determination in the present application. However, since the pre-network only has a few neurons, it cannot realize the output of the result in terms of scale, but it can be used to realize the function of feature fusion in the present application. On this basis, the neural network model synchronously inputs the first feature set, the second feature set and the third feature set to the first unit, the second unit and the third unit one by one, so as to fully spatially fuse the three feature sets through the four units connected in a three-dimensional structure, and obtain the fused image features output by the fourth unit.

[0055] Subsequently, the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain a target image. The construction position point in the target image is determined by the main network based on a preset position relationship.

[0056] It should be understood that for the above cases 1-2, the spatial fusion of features can eliminate the spatial errors such as positioning errors and attitude errors when the unmanned aerial vehicle is shooting, so as to realize more accurate marking and positioning of the construction position point.

[0057] S103, the unmanned aerial vehicle sprays and marks the actual position of the construction position point on the ground.

[0058] The unmanned aerial vehicle can map the construction position point in the target image to an actual viewfinder, and the position of the construction position point in the target image in the actual viewfinder can change with movement of the unmanned aerial vehicle. At this time, the unmanned aerial vehicle sprays and marks the ground based on the position of the construction position point in the target image in the actual viewfinder, that is, sprays and marks the position of the construction position point in the target image in the actual viewfinder, and the position of the ground sprayed and marked is the actual position of the construction position point in the target image on the ground.

[0059] In summary, the unmanned aerial vehicle can fly to the target area containing the construction position point in the BIM modeling based on the preset path in the BIM modeling, so as to take pictures of the target area at the associated at least two positions and obtain at least two images. Secondly, the unmanned aerial vehicle can fuse the at least two images through the front-end network of the neural network model to obtain fused image features, so as to realize feature fusion in the spatial dimension. In this way, the target image with the construction position point accurately labeled can be obtained by analyzing the fused image features through the main network of the neural network model. Finally, the unmanned aerial vehicle can spray and mark the actual position of the construction position point in the target image on the ground. Compared with the existing manual survey marking method, the above scheme not only has strong robustness, that is, the construction position point labeling is relatively more accurate and reliable by realizing feature fusion in the spatial dimension, but also controls the construction positioning error within ±3 cm and improves the construction efficiency by 42% compared with the traditional method.

[0060] The above is combined with Figures 1-3 The application provides a photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition. The application provides an unmanned aerial vehicle for executing the method.

[0061] The unmanned aerial vehicle is configured to:

[0062] When the unmanned aerial vehicle flies to the target area based on the preset path in the BIM modeling, the unmanned aerial vehicle takes pictures of the target area at the associated at least two positions and obtains at least two images. The target area is a region containing the construction position point in the BIM modeling. The unmanned aerial vehicle fuses the at least two images through the front-end network of the neural network model to obtain fused image features, and analyzes the fused image features through the main network of the neural network model to obtain a target image. The target image is an image in which the construction position point is labeled in the at least two images, and the construction position point in the target image corresponds to the construction position point in the BIM modeling. The unmanned aerial vehicle sprays and marks the actual position of the construction position point in the target image on the ground.

[0063] Optionally, when the UAV flies to the target area based on the preset path in the BIM modeling, the UAV photographs the target area at the associated at least two positions to obtain at least two images, including: the UAV determines to fly to the i-th target position on the preset path based on the preset path, the i-th target position corresponds to the i-th construction position point in the BIM modeling, i is an integer taking 1 to M, and M is an integer greater than 1; the UAV determines at least one associated position adjacent to the i-th target position based on the i-th target position to obtain at least two positions, the at least two positions including the i-th target position and the at least one associated position; and the UAV photographs the target area at the at least two positions respectively to obtain at least two images.

[0064] Optionally, the space in the BIM modeling is gridded into a plurality of subspaces, one subspace being a cubic region in the space, and the UAV determining to fly to the i-th target position on the preset path based on the preset path includes: the UAV determines whether the position of the UAV itself is located in the i-th target subspace in the plurality of subspaces through positioning during flight based on the preset path, the i-th target subspace being a subspace including the i-th target position; if the position of the UAV itself is located in the i-th target subspace, it indicates that the UAV flies to the i-th target position on the preset path based on the preset path; and the UAV determines at least one associated position adjacent to the i-th target position based on the i-th target position to obtain at least two positions, including: the UAV determines at least one subspace adjacent to the i-th target subspace in the plurality of subspaces based on the i-th target subspace, for any one of the at least one subspace, if the UAV flies to any position in any one of the at least one subspace, it is considered that the UAV is located in a corresponding one of the at least one associated position; and the UAV photographs the target area at the at least two positions respectively to obtain at least two images, including: the UAV photographs the target area in the i-th target subspace and the at least one subspace respectively in a direction perpendicular to the target area to obtain at least two images.

[0065] Optionally, the at least two images include a first image and a second image, the first image and the second image both contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the ith construction position point, and the reference positioning mark is associated with the plurality of construction position points in the BIM modeling; the UAV fuses the at least two images through a front-end network of the neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: the UAV respectively convolves the first image and the second image through a convolution layer of the neural network model to correspondingly obtain a first feature set and a second feature set; the UAV performs three-dimensional space fusion on the first feature set and the second feature set through the front-end network to obtain the fused image features; and the UAV analyzes the fused image features through the main network of the neural network model to obtain the target image, wherein the construction position point in the target image is determined by the main network based on the preset position relationship.

[0066] Optionally, the front-end network includes a first unit, a second unit, a third unit and a fourth unit connected in a two-dimensional structure, the connection in the two-dimensional structure means that the first unit and the second unit are connected with the third unit, and the third unit is connected with the fourth unit through a buffer; and the UAV performs three-dimensional space fusion on the first feature set and the second feature set through the front-end network to obtain the fused image features, including: the neural network model synchronously inputs the first feature set and the second feature set one by one into the first unit and the second unit; in response to the presence of input, the neural network model synchronously inputs the input of the buffer and the first feature set or the input of the buffer and the second feature set into the fourth unit to obtain the fused image features output by the fourth unit.

[0067] Optionally, the main network is a full-size fully connected layer network, and the front-end network is constructed by copying four units connected in a two-dimensional structure in the full-size fully connected layer network.

[0068] Optionally, the at least two images include a first image, a second image, and a third image, the first image, the second image, and the third image all contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the ith construction position point, and the reference positioning mark is associated with the plurality of construction position points in the BIM modeling; the unmanned aerial vehicle fuses the at least two images through a front network of the neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: the unmanned aerial vehicle convolves the first image, the second image, and the third image through a convolution layer of the neural network model to correspondingly obtain a first feature set, a second feature set, and a third feature set; the unmanned aerial vehicle fuses the first feature set, the second feature set, and the third feature set in a three-dimensional space through the front network to obtain the fused image features; and the unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain the target image, and the construction position points in the target image are determined by the main network based on the preset position relationship.

[0069] Optionally, the front network includes a first unit, a second unit, a third unit, and a fourth unit connected in a three-dimensional structure, the first unit is connected with the second unit and the third unit respectively, the second unit is connected with the first unit and the third unit respectively, the third unit is connected with the first unit and the second unit respectively, and the fourth unit is connected with the first unit, the second unit, and the third unit respectively, the unmanned aerial vehicle fuses the first feature set and the second feature set in a three-dimensional space through the front network to obtain the fused image features, including: the neural network model synchronously inputs the first feature set, the second feature set, and the third feature set into the first unit, the second unit, and the third unit one by one to obtain the fused image features output by the fourth unit.

[0070] Optionally, the main network is a full-size fully connected layer network, and the front network is constructed by copying four units connected in a three-dimensional structure in the full-size fully connected layer network.

[0071] Optionally, the unmanned aerial vehicle sprays and marks the actual positions of the construction position points in the target image on the ground, including: the unmanned aerial vehicle maps the construction position points in the target image to an actual viewfinder, and the positions of the construction position points in the target image in the actual viewfinder can change with the movement of the unmanned aerial vehicle; and the unmanned aerial vehicle sprays and marks the ground based on the positions of the construction position points in the target image in the actual viewfinder, and the positions of the ground that are sprayed and marked are the actual positions of the construction position points in the target image on the ground.

[0072] Figure 5This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Exemplarily, the electronic device may be a terminal device, or a chip (system) or other component or assembly that can be disposed in the terminal device. Figure 5 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may also include a memory 402 and / or a transceiver 403. The processor 401 is coupled to the memory 402 and the transceiver 403, for example, they can be connected via a communication bus. Alternatively, the electronic device 400 may also be a chip, such as including the processor 401; in this case, the transceiver may be the chip's input / output interface.

[0073] The following is combined with Figure 5 The various components of electronic device 400 are described in detail below:

[0074] The processor 401 is the control center of the electronic device 400. It can be a single processor or a collective term for multiple processing elements. For example, the processor 401 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0075] Optionally, the processor 401 can perform various functions of the electronic device 400, such as the aforementioned functions, by running or executing software programs stored in the memory 402 and calling scientific data stored in the memory 402. Figure 1 The method for positioning and constructing photovoltaic arrays based on UAV path planning and image recognition is shown.

[0076] In a specific implementation, as one example, processor 401 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0077] In a specific implementation, as one example, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process scientific data (such as computer programs or instructions).

[0078] The memory 402 is configured to store a software program for implementing the scheme of the present application, and the processor 401 is configured to control the execution of the software program. The specific implementation can refer to the method embodiments described above, and will not be described here.

[0079] Optionally, the memory 402 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, and can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disc storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program codes in the form of instructions or scientific data structures and capable of being accessed by a computer, but not limited to this. The memory 402 can be integrated with the processor 401, or can exist independently, and is coupled to the processor 401 through an interface circuit (not shown in the figure) of the electronic device 400. The embodiments of the present application are not limited in this regard. Figure 5

[0080] The transceiver 403 is configured to communicate with other electronic devices. For example, the electronic device 400 is a terminal device, and the transceiver 403 can be configured to communicate with a network device or another terminal device. For another example, the electronic device 400 is a network device, and the transceiver 403 can be configured to communicate with a terminal device or another network device.

[0081] Optionally, the transceiver 403 can include a receiver and a transmitter (not shown separately in the figure). The receiver is configured to implement the receiving function, and the transmitter is configured to implement the transmitting function. Figure 5

[0082] Optionally, the transceiver 403 can be integrated with the processor 401, or can exist independently, and is coupled to the processor 401 through an interface circuit (not shown in the figure) of the electronic device 400. The embodiments of the present application are not limited in this regard. Figure 5

[0083] It can be understood that, Figure 5 ​​​The structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device, and an actual electronic device can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0084] In addition, the technical effects of the electronic device 400 can refer to the technical effects of the methods described in the above method embodiments, which are not described here again.

[0085] It should be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0086] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0087] The above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions can be transferred from one website, computer, server, or scientific data center to another website, computer, server, or scientific data center through a wired (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a scientific data storage device such as a server, scientific data center, etc. that contains one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0088] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects, but can also represent an "and / or" relationship, which can be understood in the context before and after.

[0089] In this application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0090] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0091] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0093] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0094] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0095] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0096] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A photovoltaic array positioning construction method based on unmanned aerial vehicle path planning and image recognition, characterized in that, The method is applied to a UAV, and the method comprises: When the UAV flies to the air above a target area based on a preset path in BIM modeling, the UAV photographs the target area at at least two associated positions to obtain at least two images; the target area is an area containing a construction position point in the BIM modeling; The UAV fuses the at least two images through a front-end network of a neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, the target image being an image in which a construction position point is labeled in the at least two images, the construction position point in the target image corresponding to the construction position point in the BIM modeling; The UAV sprays and marks an actual position on the ground to which the construction position point in the target image is mapped.

2. The method of claim 1, wherein, When the UAV flies to the air above a target area based on a preset path in BIM modeling, the UAV photographs the target area at at least two associated positions to obtain at least two images, comprising: The UAV determines that the UAV flies to an i-th target position on the preset path based on the preset path, the i-th target position corresponding to an i-th construction position point in the BIM modeling, i being an integer taking 1 to M, and M being an integer greater than 1; The UAV determines at least one associated position adjacent to the i-th target position based on the i-th target position to obtain the at least two positions, the at least two positions comprising the i-th target position and the at least one associated position; The UAV photographs the target area at the at least two positions respectively to obtain the at least two images.

3. The method of claim 2, wherein, The space in the BIM modeling is gridded into a plurality of subspaces, one subspace being a cubic region in the space, and the UAV determining that the UAV flies to an i-th target position on the preset path based on the preset path comprises: In the process of flying based on the preset path, the UAV determines whether the position of the UAV itself is located in an i-th target subspace in the plurality of subspaces through positioning, the i-th target subspace being a subspace containing the i-th target position; if the position of the UAV itself is located in the i-th target subspace, it indicates that the UAV flies to the i-th target position on the preset path based on the preset path; And the UAV determining at least one associated position adjacent to the i-th target position based on the i-th target position to obtain the at least two positions comprises: The UAV determines at least one subspace adjacent to the i-th target subspace in the plurality of subspaces based on the i-th target subspace; for any one of the at least one subspace, if the UAV flies to any position in the any one subspace, it is considered that the UAV is located in a corresponding one of the at least one associated position; The unmanned aerial vehicle photographs the target area in the at least two positions respectively to obtain the at least two images, including: The unmanned aerial vehicle photographs the target area in the i-th target subspace and the at least one subspace respectively in a direction perpendicular to the target area to obtain the at least two images.

4. The method of claim 3, wherein, The at least two images include a first image and a second image, and the first image and the second image both contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the i-th construction position point, and the reference positioning mark is associated with a plurality of construction position points in BIM modeling. The unmanned aerial vehicle fuses the at least two images through a front network of a neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: The unmanned aerial vehicle convolves the first image and the second image through a convolution layer of the neural network model to correspondingly obtain a first feature set and a second feature set; The unmanned aerial vehicle fuses the first feature set and the second feature set in three-dimensional space through the front network to obtain the fused image features; The unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain the target image, and a construction position point in the target image is determined by the main network based on the preset position relationship.

5. The method of claim 4, wherein, The front network includes a first unit, a second unit, a third unit and a fourth unit connected in a two-dimensional structure, the first unit and the second unit are connected with the third unit, the third unit is connected with the fourth unit through a buffer, the unmanned aerial vehicle fuses the first feature set and the second feature set in three-dimensional space through the front network to obtain the fused image features, including: The neural network model synchronously inputs the first feature set and the second feature set one by one to the first unit and the second unit; in response to the input, the neural network model synchronously inputs the input of the buffer and the first feature set or the input of the buffer and the second feature set to the fourth unit, to obtain the fused image features output by the fourth unit.

6. The method of claim 5, wherein, The main network is a full-size fully connected layer network, and the front network is constructed by copying the four units connected in a two-dimensional structure in the full-size fully connected layer network.

7. The method according to any one of claims 1 to 3, characterized in that, The at least two images include a first image, a second image and a third image, and the first image, the second image and the third image all contain a reference positioning mark, the reference positioning mark satisfies a preset position relationship with the i-th construction position point, and the reference positioning mark is associated with a plurality of construction position points in BIM modeling. The unmanned aerial vehicle fuses at least two images through a front network of a neural network model to obtain fused image features, and analyzes the fused image features through a main network of the neural network model to obtain a target image, including: The unmanned aerial vehicle respectively convolves the first image, the second image and the third image through a convolution layer of the neural network model to correspondingly obtain a first feature set, a second feature set and a third feature set; The unmanned aerial vehicle fuses the first feature set, the second feature set and the third feature set through the front network to obtain the fused image features; The unmanned aerial vehicle analyzes the fused image features through the main network of the neural network model to obtain the target image, and a construction position point in the target image is determined by the main network based on the preset position relationship.

8. The method of claim 7, wherein, The front network includes a first unit, a second unit, a third unit and a fourth unit connected in a three-dimensional structure, that is, the first unit is connected with the second unit and the third unit, the second unit is connected with the first unit and the third unit, the third unit is connected with the first unit and the second unit, and the fourth unit is connected with the first unit, the second unit and the third unit, the unmanned aerial vehicle fuses the first feature set and the second feature set through the front network to obtain the fused image features, including: The neural network model synchronously inputs the first feature set, the second feature set and the third feature set one by one into the first unit, the second unit and the third unit to obtain the fused image features output by the fourth unit.

9. The method of claim 8, wherein, The main network is a full-size fully connected layer network, and the front network is constructed by copying the four units connected in a three-dimensional structure in the full-size fully connected layer network.

10. The method of claim 1, wherein, The unmanned aerial vehicle sprays and marks the actual position of the construction position point on the ground, including: The unmanned aerial vehicle maps the construction position point in the target image to an actual viewfinder, and the position of the construction position point in the target image in the actual viewfinder can change with the movement of the unmanned aerial vehicle; The unmanned aerial vehicle sprays and marks the ground based on the position of the construction position point in the target image in the actual viewfinder, and the position of the ground sprayed and marked is the actual position of the construction position point on the ground.

Images