Orthophoto generation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a method, device, electronic device and storage medium for generating orthophotos. The method includes: obtaining a digital surface model of a target area and a set of images corresponding to the target area photographed by a drone; detecting an occluded area and a visible area of ​​the digital surface model according to the set of images; performing color compensation on the occluded area and / or color updating on the visible area to generate an orthophoto. Through occlusion detection, the situation of repeated textures can be reduced, thereby making the quality of the generated orthophotos higher.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, electronic equipment and storage medium for generating orthophotos. Background Art

[0002] Orthophotos are generally obtained by inputting digital surface models and collecting picture sets through digital differential correction and other methods. However, due to the influence of elevation differences of ground objects, orthophotos of urban scenes often have anomalies such as repeated textures. These abnormal areas often appear around buildings and are caused by being blocked by buildings, which has a great impact on orthophotos of urban scenes. The existing collection of orthophotos of urban scenes is prone to repeated textures and other problems, resulting in low image quality of orthophotos. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for generating orthophotos, so as to solve the problem of low quality of the images collected due to the existing orthophoto effects on urban scenes.

[0004] In order to achieve the above object, the present invention provides a method for generating an orthophoto in a first aspect, comprising:

[0005] Obtaining a digital surface model of the target area and a set of images corresponding to the target area taken by a drone;

[0006] Detecting the occluded area and the visible area of ​​the digital surface model according to the image set;

[0007] Perform color compensation on occluded areas and / or update the colors on visible areas to generate orthophotos.

[0008] In an embodiment of the present invention, it also includes:

[0009] determining a valid region in the digital surface model corresponding to each image in the image collection;

[0010] The occluded area and visible area of ​​the digital surface model are detected based on the image collection, including:

[0011] The occluded area and visible area of ​​the digital surface model are determined according to the valid area of ​​each image in the image set.

[0012] In an embodiment of the present invention, determining a valid area in the digital surface model corresponding to each image in the image set includes:

[0013] According to the posture information of the UAV and the size of the current image, the boundary points of the image are projected onto the digital surface model to generate a projected image;

[0014] Determine whether there is an overlapping area between the projected image and the digital surface model;

[0015] When there is an overlapped area between the projection image and the digital surface model, the overlapped area is determined as a valid area.

[0016] In an embodiment of the present invention, detecting the occluded area and the visible area of ​​the digital surface model according to the image set includes:

[0017] The scanning mode is determined according to the position where the optical center of each image is projected onto the digital surface model;

[0018] Traversing the grid points of the valid area corresponding to the digital surface model in a scanning manner;

[0019] Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point;

[0020] When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it is determined that the current grid point is in the occlusion area;

[0021] When the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area.

[0022] In an embodiment of the present invention, the scanning method includes at least one of the following:

[0023] Spiral scanning method, row-first line scanning method, column-first line scanning method.

[0024] In an embodiment of the present invention, the scanning mode is determined according to the position where the optical center of each image is projected onto the digital surface model, including:

[0025] When the position where the optical center of the image is projected onto the digital surface model is within the digital surface model, determining that the scanning mode is a spiral scanning mode;

[0026] When the position where the optical center of the image is projected onto the digital surface model is not located on the digital surface model, the scanning mode is determined to be a line scanning mode with row priority or a line scanning mode with column priority.

[0027] In an embodiment of the present invention, performing color compensation on the blocked area and / or performing color updating on the visible area to generate an orthophoto includes:

[0028] When the current grid point is in the visible area, determine whether the global elevation angle corresponding to the current grid point is empty;

[0029] When the global elevation angle is empty, fill the current grid point with the color of the corresponding position in the image;

[0030] When the global elevation angle is not empty, compare the elevation angle of the current grid point with the global elevation angle;

[0031] When the elevation angle of the current grid point is smaller than the global elevation angle, the color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0032] In an embodiment of the present invention, performing color compensation on the blocked area and / or performing color updating on the visible area to generate an orthophoto includes:

[0033] When the current grid point is an occluded area, determine whether the compensated global elevation angle corresponding to the current grid point is empty;

[0034] When the compensated global elevation angle is empty, the current grid point is filled with the compensation color of the corresponding position of the current grid point in the image;

[0035] When the compensated global elevation angle is not empty, compare the elevation angle of the current grid point with the compensated global elevation angle;

[0036] When the elevation angle of the current grid point is less than the compensated global elevation angle, the compensation color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0037] A second aspect of the present invention provides an orthophoto generation device, comprising:

[0038] An acquisition module, used to acquire a digital surface model of a target area and a set of images corresponding to the target area photographed by a drone;

[0039] A detection module, used for detecting an occluded area and a visible area of ​​a digital surface model according to an image set;

[0040] The updating module is used to perform color compensation on the blocked area and / or perform color update on the visible area to generate an orthophoto.

[0041] A third aspect of the present invention provides an electronic device, which includes a processor configured to execute the above-mentioned method for generating an orthophoto.

[0042] A fourth aspect of the present invention provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned method for generating an orthophoto.

[0043] Through the above technical solution, a digital surface model and a set of images taken by a drone are obtained, and the occluded area and the visible area of ​​the digital surface model are detected according to the image set. Then, color compensation is performed on the occluded area and / or the color of the visible area is updated to reduce the situation of repeated textures, thereby making the quality of the generated orthophoto higher.

[0044] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0046] Figure 1 is a flow chart of a method for generating an orthophoto provided by an embodiment of the present invention;

[0047] Figure 2 is a flow chart of a method for generating an orthophoto provided by another embodiment of the present invention;

[0048] Figure 3 is a flow chart of a method for detecting an occluded area and a visible area provided in an embodiment of the present invention;

[0049] Figure 4 is a structural schematic diagram of the position where the optical center of an image is projected onto a digital surface model provided by an embodiment of the present invention;

[0050] FIG5( a ) is a schematic diagram of the structure of an orthophoto generated without occlusion detection in an example;

[0051] FIG5( b ) is a schematic diagram of the structure of an orthophoto image generated in an example of the present invention;

[0052] FIG6( a ) is a schematic diagram of the structure of an orthophoto generated without occlusion detection in another example;

[0053] FIG6( b ) is a schematic diagram of the structure of an orthophoto image generated in another example of the present invention;

[0054] Figure 7 is a schematic diagram of the structure of an orthophoto generating device provided by an embodiment of the present invention;

[0055] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0057] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0058] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0059] Figure 1 FIG. 1 is a flow chart of a method for generating an orthophoto according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a method for generating an orthophoto, which may include the following steps.

[0060] In step S11, a digital surface model of the target area and a set of images corresponding to the target area photographed by the drone are obtained. In an embodiment of the present invention, a digital surface model (DSM) refers to a set of digital expressions of the surface morphology of an object. DSM can be a simulation of the surface of a land object, including a ground elevation model of a vegetation surface, a house surface, etc. DSM can be used to detect the development status of a city. The processor can obtain a digital surface model of the target area, and through a set of images taken during the flight of the drone, so as to identify the occluded area and the visible area of ​​the digital surface model through a set of images from different angles.

[0061] In step S12, the occlusion area and visible area of ​​the digital surface model are detected according to the image set. In an embodiment of the present invention, after the processor obtains the image set taken by the drone at a certain angle, it can traverse each grid point on the DSM by scanning. In image editing, the grid is a pixel, which is to convert the instructions in the image into pixels. Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point. Elevation refers to the distance to the geoid along the plumb line direction, and the elevation angle is the angle between the position where the drone image is taken and the grid point. When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it is determined that the current grid point is in the occlusion area; when the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area. The occlusion area and the visible area are determined so that the current grid point can be color compensated or updated according to different areas. The occlusion area and the visible area in the DSM corresponding to the pictures taken by the drone at different angles are different. Therefore, the occlusion area and the visible area can be divided according to the specific image.

[0062] In step S13, color compensation is performed on the blocked area and / or color update is performed on the visible area to generate an orthophoto. In an embodiment of the present invention, after determining that the current grid point is in the blocked area or the visible area, it can be determined whether the current grid point is empty. If the current grid point is empty, it means that the current grid point has not been color compensated or color updated, and the color of the image corresponding to the current grid point is filled to the current grid point. In the case that the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the picture with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees and the elevation angle of the second image is 45 degrees, then the image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, then the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is compensated or updated to the current grid point. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0063] Through the above technical solution, a digital surface model and a set of images taken by a drone are obtained, and the occluded area and the visible area of ​​the digital surface model are detected according to the image set. Then, color compensation is performed on the occluded area and / or the color of the visible area is updated to reduce the situation of repeated textures, thereby making the quality of the generated orthophoto higher.

[0064] Figure 2 FIG. 1 is a flow chart of a method for generating an orthophoto according to an embodiment of the present invention. Figure 2 As shown, it may also include:

[0065] Step S14, determining a valid area in the digital surface model corresponding to each image in the image set;

[0066] Step S12, detecting the blocked area and visible area of ​​the digital surface model according to the image set, includes:

[0067] Step S121 : determining the occluded area and the visible area of ​​the digital surface model according to the valid area of ​​each image in the images.

[0068] In an embodiment of the present invention, the effective area refers to the area where the image taken by the drone overlaps with the digital surface model. Since the drone takes images from multiple angles, the image may contain the DSM area and the area outside the DSM. Therefore, before detecting the occluded area and the visible area of ​​the DSM corresponding to each image, the DSM and the effective area corresponding to each image are required. In this way, it is not necessary to traverse all the grid points of each image later, but only the grid points of the effective area need to be traversed. Work efficiency can be improved.

[0069] In an embodiment of the present invention, step S14, determining a valid area in the digital surface model corresponding to each image in the image set may include:

[0070] According to the posture information of the UAV and the size of the current image, the boundary points of the image are projected onto the digital surface model to generate a projected image;

[0071] Determine whether there is an overlapping area between the projected image and the digital surface model;

[0072] When there is an overlapped area between the projection image and the digital surface model, the overlapped area is determined as a valid area.

[0073] Specifically, the processor traverses the image set taken by the drone, and projects the boundary points of the image to the DSM model according to the posture information and image size of the drone corresponding to each image to generate a projected image. Further, it is determined whether the projected image and the DSM have overlapping areas. In the case of an overlapping area, the overlapping area is determined as a valid area and the valid area is retained. In the case where there is no overlapping area, that is, the current image does not capture the DSM, so the current image is discarded. In one example, both the DSM and the image are quadrilaterals. The four boundary vertices of the image are projected onto the plane where the DSM is located to form a quadrilateral A, and it is determined whether quadrilateral A has an intersection with the quadrilateral corresponding to the DSM. In the case of an intersection, the image is retained and regarded as a valid image, and the valid mask area is recorded, otherwise the image is discarded. In this way, the elimination of images that do not belong to the DSM can greatly improve the work efficiency of detecting the DSM occluded area and the visible area.

[0074] Figure 3 FIG. 1 is a flow chart of a method for detecting an occluded area and a visible area provided by an embodiment of the present invention. Figure 3 As shown, in an embodiment of the present invention, detecting the occluded area and the visible area of ​​the digital surface model according to the image set may include the following steps:

[0075] Step S31, determining a scanning mode according to the position where the optical center of each image is projected onto the digital surface model;

[0076] Step S32, traversing the grid points of the valid area corresponding to the digital surface model in a scanning manner;

[0077] Step S33, comparing the elevation angle of the current grid point with the elevation angle of the adjacent grid point;

[0078] Step S34: when the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, determine that the current grid point is in the shielding area;

[0079] Step S35: When the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, determine that the current grid point is in the visible area.

[0080] In the embodiment of the present invention, the optical center refers to the origin of the camera coordinate system, which is a three-dimensional rectangular coordinate system established with the focal center of the camera as the origin and the optical axis as the Z axis. The optical center projection is the position where the origin of the camera coordinate system is projected to the DSM. Different scanning modes are established according to the relative position of the optical center of the drone image corresponding to the ground projection point, that is, the relative position of the optical center projection point and the DSM range.

[0081] Figure 4 Schematic diagram of the structure of the position where the optical center of an image is projected onto a digital surface model provided by an embodiment of the present invention. Figure 4 As shown, the scanning method may include at least one of the following:

[0082] Spiral scanning method, row-first line scanning method, column-first line scanning method.

[0083] In an embodiment of the present invention, the scanning mode is determined according to the position where the optical center of each image is projected onto the digital surface model, including:

[0084] When the position where the optical center of the image is projected onto the digital surface model is within the digital surface model, determining that the scanning mode is a spiral scanning mode;

[0085] When the position where the optical center of the image is projected onto the digital surface model is not located on the digital surface model, the scanning mode is determined to be a line scanning mode with row priority or a line scanning mode with column priority.

[0086] In an embodiment of the present invention, the scanning mode may include a spiral scanning mode or a line scanning mode, and the line scanning mode may also include a line scanning mode with limited rows and a line scanning mode with priority columns. Assuming that area 0 is a DSM area, then areas 1 to 8 are all areas outside the DSM. In one example, the position where the optical center of the image is projected to the DSM is in area 0, and it can be determined that the scanning mode is a spiral scanning mode, for example, each grid point on the DSM is traversed in a clockwise circular manner. In another example, the position where the optical center of the image is projected to the DSM is in area 4 or 8, then each grid point on the DSM is traversed in a column-priority line scanning mode with the boundary point of the DSM as the starting point, and for each column of grid points, it is necessary to divide each column of grid points into two parts for expansion and traversal according to the row number where the projection point is located. In another example, the position where the optical center of the image is projected to the DSM is in area 2 or 6, then each grid point on the DSM is traversed in a row-priority line scanning mode with the boundary point of the DSM as the starting point, and for each row of grid points, it is necessary to divide each row of grid points into two parts for expansion and traversal according to the column number where the projection point is located. In another example, the position where the optical center of the image is projected onto the DSM is in regions 1, 3, 5, and 7, and each grid point on the DSM is traversed in a row-first or column-first line scanning manner. Traversing the grid points of the DSM in different scanning manners can improve work efficiency.

[0087] In an embodiment of the present invention, the grid points of the valid area corresponding to the DSM can be traversed in different scanning modes. Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point. The adjacent grid point refers to the grid point on the line connecting the current grid point and the optical center projection point, which is closest to the current grid point. In an example, calculate the set L of all grid points uploaded on the DSM by the line qP connecting the current grid point q and the optical center projection point P. Then on L, the point closest to q is the adjacent grid point. For example, the coordinates of the current grid point are (500, 400), and the coordinates of the optical center projection point are (502, 400), then the coordinates of the adjacent grid point are (501, 400). On this basis, compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point. If the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it means that the current grid point is in the blocked area, so the current grid point is invisible, and the maximum elevation angle of the current grid point is updated to the elevation angle of the adjacent grid point; if the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it means that the current grid point is in the visible area, so the current grid point is visible, and the elevation angle of the current grid point is determined to be the maximum elevation angle.

[0088] By using the above method to determine the occluded area and the visible area, color compensation or color update can be performed on each grid point, which can effectively avoid the occurrence of repeated textures.

[0089] In an embodiment of the present invention, performing color compensation on the blocked area and / or performing color updating on the visible area to generate an orthophoto includes:

[0090] When the current grid point is in the visible area, determine whether the global elevation angle corresponding to the current grid point is empty;

[0091] When the global elevation angle is empty, fill the current grid point with the color of the corresponding position in the image;

[0092] When the global elevation angle is not empty, compare the elevation angle of the current grid point with the global elevation angle;

[0093] When the elevation angle of the current grid point is smaller than the global elevation angle, the color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0094] Specifically, when the current grid point is in the visible area, that is, the current grid point is visible, it is determined whether the global elevation angle corresponding to the current grid point is empty. If it is empty, it means that the current grid point has not been updated in color, and the color of the image corresponding to the current grid point is filled to the current grid point. When the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the image with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees. The image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, and the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is updated. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0095] In an embodiment of the present invention, performing color compensation on the blocked area and / or performing color updating on the visible area to generate an orthophoto includes:

[0096] When the current grid point is an occluded area, determine whether the compensated global elevation angle corresponding to the current grid point is empty;

[0097] When the compensated global elevation angle is empty, the current grid point is filled with the compensation color of the corresponding position of the current grid point in the image;

[0098] When the compensated global elevation angle is not empty, compare the elevation angle of the current grid point with the compensated global elevation angle;

[0099] When the elevation angle of the current grid point is less than the compensated global elevation angle, the compensation color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0100] Specifically, when the current grid point is an occluded area, that is, the current grid point is invisible, it is determined whether the global elevation angle corresponding to the current grid point is empty. If it is empty, it means that the current grid point has not been color compensated, and the compensation color of the image corresponding to the current grid point is filled to the current grid point. When the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the picture with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees. The image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, and the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is compensated. The compensation color refers to the color of the visible area on the DSM that is the same as the point where the current grid point is projected to the camera. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0101] Figure 5(a) is a schematic diagram of the structure of an orthophoto generated without occlusion detection in an example; Figure 5(b) is a schematic diagram of the structure of an orthophoto generated in an example of the present invention. Repeated textures appear in the circled area of ​​Figure 5(a), while repeated textures do not appear in Figure 5(b), and the orthophoto quality is higher.

[0102] Figure 6(a) is a schematic diagram of the structure of an orthophoto generated in another example without occlusion detection; Figure 6(b) is a schematic diagram of the structure of an orthophoto generated in another example of the present invention. The circled area in Figure 6(a) has repeated textures, while Figure 6(b) does not have repeated textures, and the orthophoto quality is higher.

[0103] It can be concluded from the above examples that the embodiments of the present invention perform color compensation on the occluded area and / or update the color of the visible area through occlusion detection, thereby reducing the situation of repeated textures, thereby making the quality of the generated orthophoto higher.

[0104] Figure 7 is a schematic diagram of the structure of the device for generating orthophotos provided by an embodiment of the present invention, such as Figure 7 As shown, the present invention provides a device for generating an orthophoto, comprising:

[0105] An acquisition module 71 is used to acquire a digital surface model of a target area and a set of images corresponding to the target area photographed by a drone;

[0106] A detection module 72, configured to detect an occluded area and a visible area of ​​a digital surface model according to the image set;

[0107] The updating module 73 is used to perform color compensation on the blocked area and / or update the color on the visible area to generate an orthophoto.

[0108] In an embodiment of the present invention, a digital surface model (DSM) refers to a set of digital expressions of the surface morphology of an object. DSM can be a simulation of the surface of a ground object, including a ground elevation model of a vegetation surface, a house surface, etc. DSM can be used to detect the development status of a city. The processor can obtain a digital surface model of the target area, and through a set of images taken during the flight of the drone, so as to identify the occluded area and the visible area of ​​the digital surface model through a set of images from different angles.

[0109] In an embodiment of the present invention, after the processor obtains a set of images taken by the drone at a certain angle, it can traverse each grid point on the DSM by scanning. In image editing, the grid is a pixel, which is to convert the instructions in the image into pixels. Compare the size of the elevation angle of the current grid point with the elevation angle of the adjacent grid point. Elevation refers to the distance to the geoid along the plumb line direction, and the elevation angle is the angle between the position where the drone image is taken and the grid point. When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it is determined that the current grid point is in the occlusion area; when the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area. The occlusion area and the visible area are determined so that the color compensation or update of the current grid point is performed according to different areas. The occlusion area and the visible area in the DSM corresponding to the pictures taken by the drone at different angles are different. Therefore, the occlusion area and the visible area can be divided according to the specific image.

[0110] In an embodiment of the present invention, after determining that the current grid point is in the occluded area or the visible area, it can be determined whether the current grid point is empty. If the current grid point is empty, it means that the current grid point has not been color compensated or updated, and the color of the image corresponding to the current grid point is filled to the current grid point. In the case that the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the picture with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees, then the image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, then the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is compensated or updated to the current grid point. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0111] Through the above technical solution, a digital surface model and a set of images taken by a drone are obtained, and the occluded area and the visible area of ​​the digital surface model are detected according to the image set. Then, color compensation is performed on the occluded area and / or the color of the visible area is updated to reduce the situation of repeated textures, thereby making the quality of the generated orthophoto higher.

[0112] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 8 As shown, the present invention also provides an electronic device, which includes a processor 810, configured to execute the above-mentioned orthophoto generation method. In an embodiment of the present invention, the device may include a processor 810 and a memory 820. The memory 820 may store instructions, which, when executed by the processor 810, may enable the processor 810 to execute the orthophoto generation method described in the previous embodiment.

[0113] Specifically, in the embodiment of the present invention, the processor 810 may be configured to:

[0114] Obtaining a digital surface model of the target area and a set of images corresponding to the target area taken by a drone;

[0115] Detecting the occluded area and the visible area of ​​the digital surface model according to the image set;

[0116] Perform color compensation on occluded areas and / or update the colors on visible areas to generate orthophotos.

[0117] In an embodiment of the present invention, a digital surface model (DSM) refers to a set of digital expressions of the surface morphology of an object. DSM can be a simulation of the surface of a ground object, including a ground elevation model of a vegetation surface, a house surface, etc. DSM can be used to detect the development status of a city. The processor can obtain a digital surface model of a target area, and through a set of images taken during the flight of the drone, so as to identify the occluded area and the visible area of ​​the digital surface model through a set of images from different angles.

[0118] In an embodiment of the present invention, after the processor obtains a set of images taken by the drone at a certain angle, it can traverse each grid point on the DSM by scanning. In image editing, the grid is a pixel, which is to convert the instructions in the image into pixels. Compare the size of the elevation angle of the current grid point with the elevation angle of the adjacent grid point. Elevation refers to the distance to the geoid along the plumb line direction, and the elevation angle is the angle between the position where the drone image is taken and the grid point. When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it is determined that the current grid point is in the occlusion area; when the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area. The occlusion area and the visible area are determined so that the color compensation or update of the current grid point is performed according to different areas. The occlusion area and the visible area in the DSM corresponding to the pictures taken by the drone at different angles are different. Therefore, the occlusion area and the visible area can be divided according to the specific image.

[0119] In an embodiment of the present invention, after determining that the current grid point is in the occluded area or the visible area, it can be determined whether the current grid point is empty. If the current grid point is empty, it means that the current grid point has not been color compensated or updated, and the color of the image corresponding to the current grid point is filled to the current grid point. In the case that the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the picture with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees, then the image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, then the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is compensated or updated to the current grid point. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0120] Through the above technical solution, a digital surface model and a set of images taken by a drone are obtained, and the occluded area and the visible area of ​​the digital surface model are detected according to the image set. Then, color compensation is performed on the occluded area and / or the color of the visible area is updated to reduce the situation of repeated textures, thereby making the quality of the generated orthophoto higher.

[0121] In an embodiment of the present invention, the processor 810 may also be configured to:

[0122] determining a valid region in the digital surface model corresponding to each image in the image collection;

[0123] The occluded area and visible area of ​​the digital surface model are determined according to the valid area of ​​each image in the image set.

[0124] Specifically, the effective area refers to the area where the image taken by the drone overlaps with the digital surface model. Since the drone takes images from multiple angles, the image may contain the DSM area and the area outside the DSM. Therefore, before detecting the occluded area and visible area of ​​the DSM corresponding to each image, the DSM and the effective area corresponding to each image are required. In this way, when traversing the grid points later, there is no need to traverse all the grid points of each image, but only the grid points of the effective area need to be traversed. This can improve work efficiency.

[0125] In an embodiment of the present invention, the processor 810 may also be configured to:

[0126] Determining a valid region in the digital surface model corresponding to each image in the image collection includes:

[0127] According to the posture information of the UAV and the size of the current image, the boundary points of the image are projected onto the digital surface model to generate a projected image;

[0128] Determine whether there is an overlapping area between the projected image and the digital surface model;

[0129] When there is an overlapped area between the projection image and the digital surface model, the overlapped area is determined as a valid area.

[0130] Specifically, the processor traverses the image set taken by the drone, and projects the boundary points of the image to the DSM model according to the posture information and image size of the drone corresponding to each image to generate a projected image. Further, it is determined whether the projected image and the DSM have overlapping areas. In the case of an overlapping area, the overlapping area is determined as a valid area and the valid area is retained. In the case where there is no overlapping area, that is, the current image does not capture the DSM, so the current image is discarded. In one example, both the DSM and the image are quadrilaterals. The four boundary vertices of the image are projected onto the plane where the DSM is located to form a quadrilateral A, and it is determined whether quadrilateral A has an intersection with the quadrilateral corresponding to the DSM. In the case of an intersection, the image is retained and regarded as a valid image, and the valid mask area is recorded, otherwise the image is discarded. In this way, the elimination of images that do not belong to the DSM can greatly improve the work efficiency of detecting the DSM occluded area and the visible area.

[0131] In an embodiment of the present invention, the processor 810 may also be configured to:

[0132] The occluded area and visible area of ​​the digital surface model detected based on the image collection include:

[0133] The scanning mode is determined according to the position where the optical center of each image is projected onto the digital surface model;

[0134] Traversing the grid points of the valid area corresponding to the digital surface model in a scanning manner;

[0135] Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point;

[0136] When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it is determined that the current grid point is in the occlusion area;

[0137] When the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area.

[0138] In the embodiment of the present invention, the optical center refers to the origin of the camera coordinate system, which is a three-dimensional rectangular coordinate system established with the focal center of the camera as the origin and the optical axis as the Z axis. The optical center projection is the position where the origin of the camera coordinate system is projected to the DSM. Different scanning modes are established according to the relative position of the optical center of the drone image corresponding to the ground projection point, that is, the relative position of the optical center projection point and the DSM range.

[0139] In an embodiment of the present invention, the grid points of the valid area corresponding to the DSM can be traversed in different scanning modes. Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point. The adjacent grid point refers to the grid point on the line connecting the current grid point and the optical center projection point, which is closest to the current grid point. In an example, calculate the set L of all grid points uploaded on the DSM by the line qP connecting the current grid point q and the optical center projection point P. Then on L, the point closest to q is the adjacent grid point. For example, the coordinates of the current grid point are (500, 400), and the coordinates of the optical center projection point are (502, 400), then the coordinates of the adjacent grid point are (501, 400). On this basis, compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point. If the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, it means that the current grid point is in the blocked area, so the current grid point is invisible, and the maximum elevation angle of the current grid point is updated to the elevation angle of the adjacent grid point; if the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it means that the current grid point is in the visible area, so the current grid point is visible, and the elevation angle of the current grid point is determined to be the maximum elevation angle.

[0140] By using the above method to determine the occluded area and the visible area, color compensation or color update can be performed on each grid point, which can effectively avoid the occurrence of repeated textures.

[0141] In an embodiment of the present invention, the scanning method includes at least one of the following:

[0142] Spiral scanning method, row-first line scanning method, column-first line scanning method.

[0143] In an embodiment of the present invention, the scanning mode is determined according to the position where the optical center of each image is projected onto the digital surface model, including:

[0144] When the position where the optical center of the image is projected onto the digital surface model is within the digital surface model, determining that the scanning mode is a spiral scanning mode;

[0145] When the position where the optical center of the image is projected onto the digital surface model is not located on the digital surface model, the scanning mode is determined to be a line scanning mode with row priority or a line scanning mode with column priority.

[0146] In an embodiment of the present invention, the scanning mode may include a spiral scanning mode or a line scanning mode, and the line scanning mode may further include a row-limited line scanning mode and a column-prioritized line scanning mode. Figure 4It is a structural schematic diagram of the position where the optical center of an image is projected to a digital surface model provided by an embodiment of the present invention. Assuming that area 0 is a DSM area, then areas 1 to 8 are all areas outside the DSM. In one example, the position where the optical center of the image is projected to the DSM is in area 0, and it can be determined that the scanning mode is a spiral scanning mode, for example, each grid point on the DSM is traversed in a clockwise circular manner. In another example, the position where the optical center of the image is projected to the DSM is in area 4 or 8, then each grid point on the DSM is traversed in a column-priority line scanning manner starting from the boundary point of the DSM, and for each column of grid points, it is necessary to divide each column of grid points into two parts for expansion and traversal according to the row number where the projection point is located. In another example, the position where the optical center of the image is projected to the DSM is in area 2 or 6, then each grid point on the DSM is traversed in a row-priority line scanning manner starting from the boundary point of the DSM, and for each row of grid points, it is necessary to divide each row of grid points into two parts for expansion and traversal according to the column number where the projection point is located. In another example, the position where the optical center of the image is projected onto the DSM is in regions 1, 3, 5, and 7, and each grid point on the DSM is traversed in a row-first or column-first line scanning manner. Traversing the grid points of the DSM in different scanning manners can improve work efficiency.

[0147] In an embodiment of the present invention, the processor 810 may also be configured to:

[0148] Color compensation for occluded areas and / or color update for visible areas to generate orthophotos include:

[0149] When the current grid point is in the visible area, determine whether the global elevation angle corresponding to the current grid point is empty;

[0150] When the global elevation angle is empty, fill the current grid point with the color of the corresponding position in the image;

[0151] When the global elevation angle is not empty, compare the elevation angle of the current grid point with the global elevation angle;

[0152] When the elevation angle of the current grid point is smaller than the global elevation angle, the color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0153] Specifically, when the current grid point is in the visible area, that is, the current grid point is visible, it is determined whether the global elevation angle corresponding to the current grid point is empty. If it is empty, it means that the current grid point has not been updated in color, and the color of the image corresponding to the current grid point is filled to the current grid point. When the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the image with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees. The image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, and the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is updated. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0154] In an embodiment of the present invention, the processor 810 may also be configured to:

[0155] Color compensation for occluded areas and / or color update for visible areas to generate orthophotos include:

[0156] When the current grid point is an occluded area, determine whether the compensated global elevation angle corresponding to the current grid point is empty;

[0157] When the compensated global elevation angle is empty, the current grid point is filled with the compensation color of the corresponding position of the current grid point in the image;

[0158] When the compensated global elevation angle is not empty, compare the elevation angle of the current grid point with the compensated global elevation angle;

[0159] When the elevation angle of the current grid point is less than the compensated global elevation angle, the compensation color of the current grid point at the corresponding position of the image is filled into the current grid point.

[0160] Specifically, when the current grid point is an occluded area, that is, the current grid point is invisible, it is determined whether the global elevation angle corresponding to the current grid point is empty. If it is empty, it means that the current grid point has not been color compensated, and the compensation color of the image corresponding to the current grid point is filled to the current grid point. When the current grid point is not empty, the elevation angle of the current grid point and the global elevation angle are compared, and the image with the smaller elevation angle is taken as the target image. The global elevation angle refers to the elevation angle of the picture with the smallest elevation angle corresponding to the current grid point. For example, the elevation angle of the first image is 30 degrees, and the elevation angle of the second image is 45 degrees. The image corresponding to the elevation angle of 30 degrees is selected as the target image, and the global elevation angle at this time is 30 degrees. The elevation angle of the third image is 5 degrees, and the third image is taken as the target image, and the global elevation angle at this time is 5 degrees. Finally, the image with the smallest elevation angle among all images is taken as the image corresponding to the grid point, and the color of the corresponding point of the image is compensated. The compensation color refers to the color of the visible area on the DSM that is the same as the point where the current grid point is projected to the camera. In this way, the color of each grid point can be made more accurate to obtain a higher quality orthophoto.

[0161] Examples of processor 810 may include, but are not limited to, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and a state machine, etc. The processor may perform signal encoding, data processing, power control, input / output processing.

[0162] Examples of memory 820 may include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a processor.

[0163] The present invention also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned method for generating orthophotos.

[0164] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0170] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0171] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0172] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0173] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for generating an orthophoto, characterized in that: include: Acquire a digital surface model of a target area and a set of images corresponding to the target area taken by a drone; Detecting an occluded area and a visible area of ​​the digital surface model according to the image set; Performing color compensation on the blocked area and / or updating the color of the visible area to generate an orthophoto; The method further includes: determining a valid area in the digital surface model corresponding to each image in the image set; detecting the occluded area and the visible area of ​​the digital surface model according to the image set includes: determining the occluded area and the visible area of ​​the digital surface model according to the valid area of ​​each image in the image set; Wherein, determining the valid area in the digital surface model corresponding to each image in the image set includes: projecting the boundary points of the image to the digital surface model according to the posture information of the drone and the size of the current image to generate a projection image; judging whether the projection image and the digital surface model have an overlapping area; and if the projection image and the digital surface model have an overlapping area, determining the overlapping area as the valid area.

2. The generation method according to claim 1, characterized in that: The detecting the blocked area and the visible area of ​​the digital surface model according to the image set comprises: Determine the scanning mode according to the position where the optical center of each image is projected onto the digital surface model; Traversing the grid points of the valid area corresponding to the digital surface model according to the scanning method; Compare the elevation angle of the current grid point with the elevation angle of the adjacent grid point; When the elevation angle of the current grid point is less than or equal to the elevation angle of the adjacent grid point, determining that the current grid point is in the shielding area; When the elevation angle of the current grid point is greater than the elevation angle of the adjacent grid point, it is determined that the current grid point is in the visible area.

3. The generation method according to claim 2, characterized in that: The scanning method includes at least one of the following: Spiral scanning method, row-first line scanning method, column-first line scanning method.

4. The generation method according to claim 3, characterized in that: The step of determining the scanning mode according to the position where the optical center of each image is projected onto the digital surface model comprises: In a case where the position where the optical center of the image is projected onto the digital surface model is located within the digital surface model, determining that the scanning mode is the spiral scanning mode; When the position where the optical center of the image is projected onto the digital surface model is not located on the digital surface model, the scanning mode is determined to be the line scanning mode with row priority or the line scanning mode with column priority.

5. The generation method according to claim 2, characterized in that: The color compensation of the blocked area and / or the color update of the visible area to generate an orthophoto includes: In the case where the current grid point is a visible area, determining whether the global elevation angle corresponding to the current grid point is empty; When the global elevation angle is empty, fill the current grid point with the color of the current grid point at the corresponding position of the image; When the global elevation angle is not empty, comparing the elevation angle of the current grid point with the global elevation angle; When the elevation angle of the current grid point is smaller than the global elevation angle, the color of the current grid point at the corresponding position of the image is filled into the current grid point.

6. The generation method according to claim 2, characterized in that: The color compensation of the blocked area and / or the color update of the visible area to generate an orthophoto includes: In the case where the current grid point is an occluded area, determining whether the compensated global elevation angle corresponding to the current grid point is empty; When the compensated global elevation angle is empty, fill the current grid point with the compensation color of the current grid point at the corresponding position of the image; When the compensated global elevation angle is not empty, comparing the elevation angle of the current grid point with the compensated global elevation angle; When the elevation angle of the current grid point is smaller than the compensated global elevation angle, the current grid point is filled with the compensation color of the current grid point at the corresponding position of the image.

7. A device for generating an orthophoto, characterized in that: include: An acquisition module, used to acquire a digital surface model of a target area and a set of images corresponding to the target area photographed by a drone; A detection module, configured to detect an occluded area and a visible area of ​​the digital surface model according to the image set; An updating module, used for performing color compensation on the blocked area and / or color updating on the visible area to generate an orthophoto; The generating device is further used to: determine a valid area in the digital surface model corresponding to each image in the image set; Detecting the occluded area and the visible area of ​​the digital surface model according to the image set includes: determining the occluded area and the visible area of ​​the digital surface model according to the valid area of ​​each image in the image set; Among them, the detection module is also used to: project the boundary points of the image to the digital surface model according to the posture information of the drone and the size of the current image to generate a projection image; determine whether the projection image and the digital surface model have overlapping areas; if the projection image and the digital surface model have overlapping areas, determine the overlapping areas as the valid area.

8. An electronic device, characterized in that: The electronic device comprises a processor configured to execute the method for generating an orthophoto according to any one of claims 1 to 6.

9. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the method for generating an orthophoto according to any one of claims 1 to 6.

Images