Orthophoto generation method, device, electronic device and readable storage medium

Abstract

This application provides an orthophoto generation method, apparatus, electronic device, and readable storage medium, relating to the field of data processing technology. The method includes: projecting each original image of a target scene onto a target projection polygon (DSM) of the target scene to obtain the target projection polygon corresponding to each original image; for each original image, generating a first image set including a first initial orthophoto image corresponding to the original image based on the original image, the target projection polygon corresponding to the original image, and the DSM, wherein the color values ​​in the first initial orthophoto image are calculated from the spatial positions of grid points in the DSM located within the target projection polygon and the original image; and generating a target orthophoto image of the target scene based on each first initial orthophoto image. Therefore, by generating the first initial orthophoto image under the constraint of the target projection polygon, the required hardware resources can be reduced, thereby enabling rapid generation of the target orthophoto image.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an orthophoto generation method, device, electronic device and readable storage medium. Background Art

[0002] In the field of aerial mapping, an orthophoto generation algorithm based on aerial images is used to generate an orthophoto of a specific area based on the aerial images and digital surface models (DSM) of the specific area. However, current orthophoto algorithms generally require a large amount of hardware resources. When the available hardware resources are relatively limited, the generation of orthophotos will be affected. For example, the speed of running the algorithm is relatively slow. Therefore, how to reduce the hardware resources required by the algorithm has become a technical problem that technicians in this field need to solve urgently. Summary of the invention

[0003] The embodiments of the present application provide an orthophoto generation method, device, electronic device and readable storage medium, which can quickly generate a target orthophoto of a target scene by using the original image of the target scene when hardware resources are limited.

[0004] The embodiment of the present application can be implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for generating an orthophoto, the method comprising:

[0006] Obtain the original image set and digital surface model DSM of the target scene;

[0007] By projecting each original image in the original image set, a target projection polygon corresponding to each original image on the DSM is obtained;

[0008] For each original image, a first image set of the original image is generated according to the original image, the target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthoimage corresponding to the original image, and the color value in the first initial orthoimage is calculated by the spatial position of the grid point of the DSM located in the target projection polygon and the original image;

[0009] A target orthophoto of the target scene is generated according to the first initial orthophoto images in each first image set.

[0010] In a second aspect, an embodiment of the present application provides an orthophoto generating device, the device comprising:

[0011] The acquisition module is used to obtain the original image set and digital surface model DSM of the target scene;

[0012] A projection module, configured to obtain a target projection polygon corresponding to each original image on the DSM by projecting each original image in the original image set;

[0013] A processing module, for generating, for each original image, a first image set of the original image according to the original image, a target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthoimage corresponding to the original image, and a color value in the first initial orthoimage is calculated by the spatial position of a grid point of the DSM located within the target projection polygon and the original image;

[0014] The processing module is further used to generate a target orthophoto of the target scene according to the first initial orthophoto images in each first image set.

[0015] In a third aspect, the present invention provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the orthophoto generation method described in the aforementioned embodiment.

[0016] In a fourth aspect, the present invention provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the orthophoto generation method as described in the aforementioned embodiment.

[0017] The orthophoto image generation method, device, electronic device and readable storage medium provided by the embodiments of the present application, when the original image set and DSM of the target scene are obtained, each original image in the original image set is projected to obtain the target projection polygon corresponding to each original image on the DSM; then for each original image, a first image set of the original image is generated according to the original image, the target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthophoto image corresponding to the original image, and the color value in the first initial orthophoto image is calculated by the spatial position of the grid point of the DSM located within the target projection polygon and the original image; finally, the target orthophoto image of the target scene can be generated according to the first initial orthophoto image in each first image set. In this way, by pre-projecting the original image, the target projection polygon corresponding to the original image on the DSM can be determined, and then the first initial orthophoto image is generated with the target projection polygon as a constraint, rather than generating the first initial orthophoto image based on the entire DSM. This method can reduce the hardware resources required when generating the first initial orthophoto image, and reduce the hardware resources required when generating the target orthophoto image based on the first initial orthophoto image. Therefore, when the available hardware resources (such as memory resources and computing power) are limited, the original image of the target scene can be used to quickly generate the target orthophoto image of the target scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a flow chart of an orthophoto generation method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the grid points that need to be accessed to generate a DOM of an original image;

[0022] Figure 4 A schematic diagram of a target orthophoto provided in an embodiment of the present application;

[0023] Figure 5 for Figure 2 A schematic flow chart of the sub-steps included in step S120;

[0024] Figure 6 A schematic elevation diagram provided for an embodiment of the present application;

[0025] Figure 7 A schematic diagram of a projection polygon provided in an embodiment of the present application;

[0026] Figure 8 for Figure 5 A schematic flow chart of the sub-steps included in sub-step S122;

[0027] Fig. 9 for Figure 2 A schematic flow chart of the sub-steps included in step S130;

[0028] Fig.10 for Figure 2 A schematic flow chart of the sub-steps included in step S140;

[0029] Fig.11 A block diagram of an orthophoto generating device provided in an embodiment of the present application.

[0030] Icon: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication unit; 200 - orthophoto generating device; 210 - acquisition module; 220 - projection module; 230 - processing module. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0032] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0033] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0034] Most of the current orthophoto algorithms are applied to devices with more hardware resources, such as PC (Personal Computer). When the algorithm is applied to devices with relatively limited available hardware resources (such as embedded devices), the algorithm cannot generate orthophotos normally, which affects the generation of orthophotos in specific areas. For example, the algorithm cannot run, or the algorithm runs very slowly, resulting in a long time to obtain orthophotos.

[0035] In order to alleviate the above situation, the embodiments of the present application provide an orthophoto generation method, device, electronic device and readable storage medium, so that when the available hardware resources (such as memory resources and computing power) are limited, the original image of the target scene can be used to quickly generate a target orthophoto of the target scene.

[0036] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] Please refer to Figure 1 , Figure 1 A block diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may be, but is not limited to, a computer, a server, etc. with more available hardware resources, or an embedded device with relatively limited available hardware resources. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, the processor 120, and the communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0038] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0039] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, the memory 110 stores an orthophoto image generation device 200, which includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running software programs and modules stored in the memory 110, such as the orthophoto image generation device 200 in the embodiment of the present application, that is, the orthophoto image generation method in the embodiment of the present application is realized.

[0040] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through a network, and to send and receive data through the network.

[0041] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0042] Please refer to Figure 2 , Figure 2 The flowchart of the orthophoto image generation method provided in the embodiment of the present application is shown in FIG. The method can be applied to the electronic device 100 described above. The specific flow of the orthophoto image generation method is described in detail below. The method may include steps S110 to S140.

[0043] Step S110, obtaining an original image set and a DSM of the target scene.

[0044] In this embodiment, the original image set may include at least one original image, which may be an image obtained by a shooting device shooting the ground in the air, for example, an image obtained by performing drone aerial photography of the target scene. The number of original images in the original image set can be determined according to actual needs and is not specifically limited here. The electronic device 100 can obtain the original image set of the target scene by shooting or receiving files sent by other devices.

[0045] The electronic device 100 may also obtain the DSM (Digital Surface Model) of the target scene. DSM refers to a ground elevation model that includes the heights of surface buildings, bridges, and trees. The pixel value of a grid point in the DSM is the elevation value of the grid point. Optionally, the electronic device 100 may generate the DSM of the target scene based on the original image set. The electronic device 100 may also obtain the DSM from other devices or in other ways, which are not specifically limited here.

[0046] Step S120 , projecting each original image in the original image set to obtain a target projection polygon corresponding to each original image on the DSM.

[0047] In this embodiment, for each original image in the original image set, the original image may be projected to obtain a target projection polygon corresponding to the original image on the DSM. That is, by projecting an original image, a target projection polygon corresponding to the original image is obtained, and the target projection polygon is located on the DSM.

[0048] Step S130 , for each original image, generating a first image set of the original image according to the original image, the target projection polygon corresponding to the original image, and the DSM.

[0049] When a target projection polygon of an original image is obtained, the two-dimensional coordinates and elevation values ​​of each grid point in the target projection polygon in the DSM can be obtained according to the target projection polygon and the DSM, that is, the spatial position of each grid point in the target projection polygon is obtained. Then, the first image set of the original image can be generated according to the spatial position of each grid point in the target projection polygon and the original image.

[0050] The first image set includes a first initial orthoimage of the original image. The color values ​​in the first initial orthoimage are calculated from the spatial positions of the grid points of the DSM located within the target projection polygon and the original image, that is, calculated from the spatial positions of each grid point in the target projection polygon and the original image.

[0051] The first initial orthophoto of an original image is generated with the target projection polygon as a constraint; that is, it is generated based on the spatial positions of the grid points in the DSM that are located in the target projection polygon. In the case of generating an orthophoto based on multiple original images, the target projection polygon corresponding to an original image is only a part of the DSM and is not the same size as the DSM. That is to say, in this embodiment, the first initial orthophoto of an original image is not generated based on the spatial positions of all the grid points in the DSM, but is generated based on the spatial positions of a part of the grid points in the DSM, thereby reducing the hardware resources required for generating the first initial orthophoto.

[0052] It can be seen that in this embodiment, the memory size and calculation time occupied by the first initial orthoimage depend on the size of the target projection polygon rather than the size of the DSM, thereby reducing memory usage and shortening calculation time.

[0053] For example, Figure 3 As shown, assuming that the size of the DSM is m*n, and the size of a target projection polygon is m′*n′. Assuming that the time to access a grid point and perform calculations is Δt, when a first initial orthophoto image is generated based on all grid points of the DSM, the running speed of the algorithm is: m*n*Δt; and when the above method provided by the embodiment of the present application is adopted, the running speed is: m′*n′*Δt. It can be seen that the embodiment of the present application can increase the speed of generating the first initial orthophoto image, and then increase the speed of generating the target orthophoto image.

[0054] Furthermore, assuming that the memory occupied by the data of a grid point is ΔB, when a first initial orthophoto image is generated according to all the grid points of the DSM, the memory occupied by the algorithm is: m*n*ΔB; and when the above method provided by the embodiment of the present application is adopted, the memory occupied is: m′*n′*ΔB. It can be seen that the embodiment of the present application can reduce the memory occupied when generating the first initial orthophoto image, thereby reducing the memory resources required when generating the target orthophoto image.

[0055] It is worth noting that the target projection polygons corresponding to all the original images can be obtained first, and then the corresponding first initial orthophoto images can be generated based on the target projection polygons; or after each target projection polygon corresponding to an original image is obtained, the corresponding first initial orthophoto image can be generated based on the target projection polygon. Of course, it can also be performed in other orders, which are not specifically limited here, as long as the first initial orthophoto images of each original image are obtained based on the target projection polygon.

[0056] Step S140: generating a target orthophoto of the target scene according to the first initial orthophoto images in each first image set.

[0057] When the first initial orthophoto images of the original images are obtained, any method, such as image fusion, can be used to generate the target orthophoto image of the target scene according to the first initial orthophoto images corresponding to the original image set. Figure 4 shown.

[0058] The embodiment of the present application determines the target projection polygon corresponding to the original image on the DSM by pre-projecting the original image, and then generates the first initial orthophoto image with the target projection polygon as a constraint, rather than generating the first initial orthophoto image based on the entire DSM. This method can reduce the hardware resources required when generating the first initial orthophoto image, and reduce the hardware resources required when generating the target orthophoto image based on the first initial orthophoto image. Therefore, when the available hardware resources (for example, memory resources and computing power) are limited, the original image of the target scene can be used to quickly generate the target orthophoto image of the target scene.

[0059] Optionally, as an optional implementation, Figure 5 The target projection polygon is obtained in the manner shown. Figure 5 , Figure 5 for Figure 2 Schematic diagram of the flow of sub-steps included in step S120. Step S120 may include sub-step S121 and step S122.

[0060] Sub-step S121, determining a target elevation plane according to the elevation value in the DSM.

[0061] In this embodiment, the elevation values ​​included in the DSM may be analyzed to determine the target elevation value. When the target elevation value is determined, a target elevation plane may be established, and the elevation value of the target elevation plane is the target elevation value.

[0062] By comparison, the maximum and minimum elevation values ​​in the DSM can be determined. Figure 6As shown, the plane corresponding to the maximum elevation value is the maximum elevation plane P, and the plane corresponding to the minimum elevation value is the minimum elevation plane Q. The projection area of ​​the camera model on the maximum elevation plane P is smaller than the projection area of ​​the camera model on the minimum elevation plane Q. It can be seen that as the elevation value increases, the number of grid points that can be obtained on the DSM will decrease. When fewer grid points are obtained, the final generated target orthophoto will have blurred edges. Optionally, the target elevation value is less than the maximum elevation value in the DSM and not less than the minimum elevation value in the DSM, thereby avoiding the situation where very few grid points are obtained on the DSM, and the grid points obtained are related to the original image.

[0063] Optionally, the minimum elevation value of the DSM may also be determined as the target elevation value. Figure 6 The minimum elevation plane P in is the target elevation plane. In this way, it is ensured that all DSM grid points associated with the original images in the original image set are obtained as much as possible.

[0064] Sub-step S122, according to the posture information corresponding to each original image, each original image is projected onto the target elevation plane to obtain the projection polygon corresponding to each original image, and according to the projection polygon corresponding to each original image, the target projection polygon corresponding to the original image is obtained.

[0065] When the elevation plane is determined, for each original image, the original image can be projected onto the target elevation plane according to the posture information corresponding to the original image, thereby obtaining the projection polygon corresponding to the original image on the target elevation plane. The posture information corresponding to the original image can be the position and posture information of the shooting device when the original image is obtained by shooting. For example, when the original image is obtained by drone aerial photography, the posture information of the drone at that time can be used as the posture information corresponding to the original image.

[0066] Optionally, the back-projection positions of the four corner points of the original image on the target elevation plane can be calculated according to the position information corresponding to the original image, and the polygon formed by the four back-projection positions is the projection polygon. Thus, the projection polygon can be quickly obtained.

[0067] The back-projection positions of the four corner points can be determined by: Figure 7 As shown, according to the pose information of an original image, the position C of the camera optical center and the position N of a corner point are obtained (the position N of the corner point is not in Figure 7 ), calculate the intersection of ray CN and target elevation plane, which is the back-projection position of the corner point on the target elevation plane. After the four back-projection positions are connected, the projection polygon of the original image can be obtained.

[0068] In the case of obtaining a projection polygon of an original image, as an optional implementation, the target polygon corresponding to the projection polygon in the DSM can be used as the target projection polygon corresponding to the original image. The two-dimensional coordinates of the target polygon and the projection polygon are the same, that is, the actual area corresponding to the target polygon and the projection polygon is the same. In this way, the target projection polygon of each original image can be obtained.

[0069] In the case of obtaining a projection polygon, as another optional implementation, it is possible to Figure 8 The target projection polygon is obtained in the manner shown. Figure 8 , Figure 8 for Figure 5 The flowchart of the sub-steps included in sub-step S122 is shown in FIG.

[0070] Sub-step S1221, establishing a DSM polygon on the target elevation plane according to the size of the DSM.

[0071] That is, according to the range of the actual scene corresponding to the DSM, the DSM polygon is established on the target elevation plane. It is worth noting that when calculating the target projection polygon corresponding to the original image set, since the DSM polygon used is the same, the action of establishing the DSM polygon can be performed only once.

[0072] Sub-step S1222, calculates the overlapping part of the projection polygon corresponding to each original image and the DSM polygon, and determines the target polygon corresponding to the overlapping part in the DSM according to the position of the overlapping part corresponding to each original image in the DSM polygon, so as to obtain the target projection polygon corresponding to the original image.

[0073] When the DSM polygon and the projection polygon of an original image are determined, the intersection polygon Qi' (ie Figure 7 That is, for the two 2D polygons, the intersection between the DSM polygon and the projection polygon is calculated, thereby obtaining the intersection polygon Qi'. Thus, the overlapped part between the DSM polygon and the projection polygon of an original image can be obtained.

[0074] Then, according to the position of the overlapped portion in the DSM polygon, a target polygon corresponding to the overlapped portion in the DSM can be determined, and the target polygon can be used as the target projection polygon corresponding to the original image. The two-dimensional coordinates of the target polygon can be the same as the two-dimensional coordinates of the overlapped portion in the DSM polygon.

[0075] In this way, the target projection polygon of each original image can be obtained, and the target projection polygon does not exceed the scope of the DSM, thereby avoiding adverse effects on subsequent target orthophotos due to the target projection polygon exceeding the scope of the DSM, such as errors, resulting in the inability to generate target orthophotos, and also avoiding waste of resources due to the calculation of some unnecessary data.

[0076] In this embodiment, Fig. 9 The first initial orthophoto image is generated in the manner shown. Fig. 9 , Fig. 9 for Figure 2 Schematic diagram of the flow of sub-steps included in step S130. Step S130 may include sub-step S131 and sub-step S132.

[0077] Sub-step S131, obtaining a projection matrix corresponding to the original image according to the posture information corresponding to the original image.

[0078] The projection matrix corresponding to the original image, that is, the projection matrix of the camera when shooting the original image, can be generated according to the posture information and the camera-related parameters of the shooting device.

[0079] Sub-step S132, calculating the color value of each pixel in the first initial orthoimage according to the projection matrix, the spatial position of the grid points in the target projection polygon corresponding to the original image and the original image.

[0080] Optionally, to avoid the influence of invalid elevation values ​​in the target projection polygon, the grid points with valid elevation values ​​in the target projection polygon can be first determined, and then the color value of each pixel in the first initial orthoimage is calculated based on the two-dimensional coordinates and elevation values ​​(i.e., spatial positions) of these grid points in the DSM, combined with the projection matrix and the corresponding original image. Wherein, the grid point has an invalid elevation value, which can indicate that there is no height information of the grid point in the DSM.

[0081] The first initial orthoimage can be calculated according to the following formula: c=I(PX), where c represents the color value, I represents the original image, P represents the projection matrix, and X represents the spatial position of a grid point in the target projection polygon.

[0082] Among them, the size of the first initial orthoimage is the same as the size of the target projection polygon. Optionally, when the coordinate system used by the first initial orthoimage is different from the coordinate system of the DSM, in order to facilitate the subsequent fusion based on multiple first initial orthoimages, the position of a certain point (i.e., reference point) corresponding to the first initial orthoimage on the DSM can also be calculated. This point can be a point on the first initial orthoimage (for example, the center point) or a point outside the first initial orthoimage. For example, calculate the position of the upper left corner vertex pi of the rectangle where the first initial orthoimage is located on the DSM, that is, calculate Figure 7 The two-dimensional coordinates of the upper left corner vertex pi of the rectangle where the pre-projection area Qi' is located on the DSM.

[0083] Optionally, in this embodiment, the first image set may also include a first initial weighted image. For each original image, a first initial weighted image corresponding to the original image may be generated according to the original image, the target projection polygon corresponding to the original image, and the DSM. The pixel value of each pixel in the first initial weighted image represents a weight value, and the first initial weighted image is used to achieve color fusion of multiple first initial orthoimages to eliminate the influence of the stitching line.

[0084] In this embodiment, when generating the first initial weighted image for each original image, the pixel value of each pixel point in the first initial weighted image corresponding to the original image can be calculated based on the distance from the pixel point corresponding to each grid point within the target projection polygon corresponding to the original image in the original image to the center pixel point of the original image.

[0085] That is, the weight value is calculated based on the distance from the pixel point in the original image to the central pixel point of the original image. The pixel point in the original image corresponding to any grid point in the target projection polygon can be determined based on the projection matrix used when calculating the first initial orthophoto image corresponding to the original image and the spatial position of the grid point. The size of the first initial weight image is the same as the size of the target projection polygon.

[0086] Optionally, as an optional implementation, the larger the pixel value in the first initial weight image is, the closer the distance between the corresponding pixel point and the central pixel point of the original image is.

[0087] As a possible implementation method, the weight value in the first initial weight image can be calculated by the following formula: w=255(1-d / γ), γ is the farthest distance from the image boundary corner point to the center of the original image, and d represents the distance from a pixel point to the center in the original image.

[0088] When the first image sets corresponding to the original images are obtained, the target orthophoto image may be generated by fusion according to the first initial weighted image and the first initial orthophoto image in each first image set.

[0089] As a possible implementation, Fig.10 The target orthophoto is generated in the manner shown. Fig.10 , Fig.10 for Figure 2 Schematic diagram of the flow of sub-steps included in step S140. Step S140 may include sub-steps S141 to S144.

[0090] Sub-step S141 : for each first initial orthoimage, establishing a first Laplacian pyramid corresponding to the first initial orthoimage.

[0091] For each first initial orthoimage, the first initial orthoimage can be used as the bottom layer image to establish a first Laplacian pyramid. i represents the identifier of the original image corresponding to the first initial orthophoto image, and k represents the level of the image. The higher the level of the image, the smaller the image and the lower the resolution. The number of image layers in the first Laplacian pyramid can be set according to actual needs, for example, set to 6 layers.

[0092] To facilitate subsequent fusion, the number of image layers in the first Laplacian pyramid corresponding to each first initial orthoimage may be the same.

[0093] Sub-step S142 : for each first initial weight image, establishing a first Gaussian pyramid corresponding to the first initial weight image.

[0094] For each first initial weight image, the first initial weight image can be used as the bottom layer image to establish a first Gaussian pyramid. i represents the identifier of the original image corresponding to the first initial orthoimage, and k represents the level of the image. The number of image layers in the first Gaussian pyramid may be the same as the number of image layers in the first Laplacian pyramid.

[0095] Sub-step S143: performing layer-by-layer fusion on the images of each layer in the first Laplacian pyramid according to the images of each layer in the first Gaussian pyramid to obtain a layer-by-layer fusion result.

[0096] When the first Laplacian pyramid and the first Gaussian pyramid corresponding to each original image have been established, a layered fusion result can be obtained by performing layered fusion on each layer, wherein the number of the layered fusion results is the same as the number of image layers in the first Laplacian pyramid.

[0097] The hierarchical fusion method is as follows: an image of a certain level is obtained from each first Laplacian pyramid, and an image of the level is obtained from each first Gaussian pyramid, thereby obtaining the color value and the corresponding weight value of the image of the level of the first Laplacian pyramid. Afterwards, when the pixel point is closer to the central pixel point and the weight value is larger, the weight of the color value at the same position can be compared to determine the color value with a larger weight value (or the maximum weight value) as the color value of each corresponding pixel point in the hierarchical fusion result of the level.

[0098] Sub-step S144, merging the obtained layered fusion results to obtain the target orthophoto.

[0099] As a possible implementation, sub-step S133 and sub-step S134 may be completed in the following manner.

[0100] Before performing specific fusion, a second image set may be generated, which may include a second initial orthoimage and a second initial weighted image of the same size as the DSM. The pixel values ​​in the second initial orthoimage and the second initial weighted image may both be preset values, for example, both are 0.

[0101] Optionally, according to the size of the DSM, a DOM memory of the same size may be opened up, and then a second initial orthophoto image and a second initial weighted image corresponding to the size of the DSM may be established.

[0102] The second initial orthophoto image can be used as the bottom layer image to build the second Laplacian pyramid The second initial weight image is used as the bottom layer image to build the second Gaussian pyramid The number of image layers in the second Laplacian pyramid is the same as the number of image layers in the first Laplacian pyramid.

[0103] The first Laplacian pyramid of each original image can be and the first Gaussian pyramid Fusion to the second Laplacian pyramid and the second Gaussian pyramid In order to obtain the hierarchical fusion result.

[0104] During fusion, each layer may be fused in sequence.

[0105] For example, when performing the a-th layer fusion, the first Laplacian pyramid Get the a-th layer image And from each first Gaussian pyramid Get the a-th layer image And from the second Laplace pyramid Get the a-th layer image And from the second Gaussian pyramid Get the a-th layer image

[0106] Next, the image Fusion to to get the updated At the same time, Fusion to to get the updated Since this is the first time to integrate, the latest The initial value is 0, then Each weight value in will not be less than The weight values ​​of each weight value in , so the image can be directly Fill the image with the color values ​​of each grid At the corresponding grid in the image Fill in the weight values ​​of each grid in To complete the and Among them, it is worth mentioning that Not included In the case of the grids corresponding to all the grids in the fusion and When and Update all grids.

[0107] Next, the image Fusion to the current to get the updated At the same time, Fusion to the current to get the updated At this time, the current In the case where the values ​​are not all initial values, in this case, Need to be based on the image If the weight value of a grid is not the initial value, the weight value and image are updated. The weight value of the grid corresponding to the grid is compared with that of the grid. If the image The weight value of the grid corresponding to the grid is larger, then The color value of a grid in the image is updated The color value of the grid corresponding to the grid in the image The weight value of the grid corresponding to the grid is updated to Corresponding to the grid. If the image If the weight value of the grid corresponding to the grid is small, it is not based on the image. and The certain grid is updated.

[0108] If the weight value currently held by a grid is an initial value, it can be directly based on the image and The certain grid is updated.

[0109] Then, for the other images of the original image set corresponding to the ath layer that have not been fused, the current and To update. Among them, the last obtained current That is the hierarchical fusion combination of the ath layer.

[0110] The above hierarchical fusion process can be expressed by the following formula:

[0111]

[0112]

[0113] Here, (x,y) is used to represent a raster with row x and column y.

[0114] After completing the layered fusion, the layered fusion results can be combined according to the following formula to obtain the target orthophoto:

[0115] In order to execute the corresponding steps in the above embodiments and various possible methods, an implementation method of an orthophoto generating device 200 is given below. Optionally, the orthophoto generating device 200 can adopt the above Figure 1 The device structure of the electronic device 100 is shown. Fig.11 , Fig.11 The block diagram of the orthophoto image generation device 200 provided in the embodiment of the present application. It should be noted that the basic principle and technical effect of the orthophoto image generation device 200 provided in this embodiment are the same as those of the above embodiment. For the sake of brief description, the parts not mentioned in this embodiment can refer to the corresponding contents in the above embodiment. The orthophoto image generation device 200 may include: an acquisition module 210, a projection module 220 and a processing module 230.

[0116] The acquisition module 210 is used to obtain the original image set and DSM of the target scene.

[0117] The projection module 220 is used to obtain a target projection polygon corresponding to each original image on the DSM by projecting each original image in the original image set.

[0118] The processing module 230 is used to generate a first image set of the original image according to the original image, the target projection polygon corresponding to the original image and the DSM for each original image. The first image set includes a first initial orthoimage corresponding to the original image, and the color value in the first initial orthoimage is calculated by the spatial position of the grid point of the DSM located in the target projection polygon and the original image;

[0119] The processing module 230 is further configured to generate a target orthoimage of the target scene according to the first initial orthoimages in each first image set.

[0120] Optionally, in this embodiment, the projection module 220 is specifically used to: determine a target elevation plane according to the elevation value in the DSM, wherein the target elevation value of the target elevation plane is less than the maximum elevation value in the DSM and not less than the minimum elevation value in the DSM; project each original image onto the target elevation plane according to the posture information corresponding to each original image to obtain a projection polygon corresponding to each original image, and obtain a target projection polygon corresponding to the original image according to the projection polygon corresponding to each original image.

[0121] Optionally, in this embodiment, the projection module 220 is specifically used to: establish a DSM polygon on the target elevation plane according to the size of the DSM; calculate the overlapping part of the projection polygon corresponding to each original image and the DSM polygon, and determine the target polygon corresponding to the overlapping part in the DSM according to the position of the overlapping part corresponding to each original image in the DSM polygon, so as to obtain the target projection polygon corresponding to the original image, wherein the target projection polygon is the target polygon.

[0122] Optionally, in this embodiment, the target elevation value is the minimum elevation value.

[0123] Optionally, in this embodiment, the processing module 230 is specifically used to: obtain the projection matrix corresponding to the original image according to the posture information corresponding to the original image; calculate the color value of each pixel in the first initial orthoimage according to the projection matrix, the spatial position of the grid points in the target projection polygon corresponding to the original image and the original image.

[0124] Optionally, in this embodiment, the first image set also includes a first initial weighted image, and the processing module 230 is further used to calculate the pixel value of each pixel in the first initial weighted image corresponding to the original image according to the distance from the pixel corresponding to each grid point in the target projection polygon corresponding to the original image to the center pixel of the original image. The processing module 230 is specifically used to generate the target orthoimage according to the first initial orthoimage and the first initial weighted image in each first image set.

[0125] Optionally, in this embodiment, the processing module 230 is specifically used to: establish a first Laplacian pyramid corresponding to each first initial orthoimage for each first initial orthoimage; establish a first Gaussian pyramid corresponding to each first initial weighted image for each first initial weighted image; perform hierarchical fusion on each layer of images in the established first Laplacian pyramid according to each layer of images in the first Gaussian pyramid to obtain a hierarchical fusion result, wherein the number of the hierarchical fusion results is the same as the number of image layers in the first Laplacian pyramid; and merge the obtained hierarchical fusion results to obtain the target orthoimage.

[0126] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown in the figure may be fixed in the operating system (OS) of the electronic device 100 and may be Figure 1 Meanwhile, the data and program codes required for executing the above modules may be stored in the memory 110.

[0127] The embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and the computer program implements the orthophoto generating method when executed by a processor.

[0128] In summary, the embodiments of the present application provide an orthophoto generation method, device, electronic device and readable storage medium. When an original image set and a DSM of a target scene are obtained, each original image in the original image set is projected to obtain a target projection polygon corresponding to each original image on the DSM; then, for each original image, a first image set of the original image is generated according to the original image, the target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthophoto image corresponding to the original image, and the color value in the first initial orthophoto image is calculated by the spatial position of the grid point of the DSM located within the target projection polygon and the original image; finally, a target orthophoto image of the target scene can be generated according to the first initial orthophoto image in each first image set. In this way, by pre-projecting the original image, the target projection polygon corresponding to the original image on the DSM can be determined, and then the first initial orthophoto image is generated with the target projection polygon as a constraint, rather than generating the first initial orthophoto image based on the entire DSM. This method can reduce the hardware resources required when generating the first initial orthophoto image, and reduce the hardware resources required when generating the target orthophoto image based on the first initial orthophoto image. Therefore, when the available hardware resources (such as memory resources and computing power) are limited, the original image of the target scene can be used to quickly generate the target orthophoto image of the target scene.

[0129] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0131] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] The above description is only an optional embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for generating an orthophoto, characterized in that: The method comprises: Obtain the original image set and digital surface model DSM of the target scene; By projecting each original image in the original image set, a target projection polygon corresponding to each original image on the DSM is obtained; For each original image, a first image set of the original image is generated according to the original image, the target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthoimage corresponding to the original image, and the color value in the first initial orthoimage is calculated by the spatial position of the grid point of the DSM located in the target projection polygon and the original image; Generating a target orthophoto of the target scene according to the first initial orthophoto images in each first image set; The step of projecting each original image in the original image set to obtain a target projection polygon corresponding to each original image on the DSM includes: Determine a target elevation plane according to the elevation value in the DSM, wherein the target elevation value of the target elevation plane is less than the maximum elevation value in the DSM and not less than the minimum elevation value in the DSM; According to the posture information corresponding to each original image, each original image is projected onto the target elevation plane to obtain the projection polygon corresponding to each original image, and according to the projection polygon corresponding to each original image, the target projection polygon corresponding to the original image is obtained.

2. The method according to claim 1, characterized in that The step of obtaining a target projection polygon corresponding to each original image according to the projection polygon corresponding to the original image includes: Establishing a DSM polygon on the target elevation plane according to the size of the DSM; The overlapping part of the projection polygon corresponding to each original image and the DSM polygon is calculated, and according to the position of the overlapping part corresponding to each original image in the DSM polygon, the target polygon corresponding to the overlapping part in the DSM is determined to obtain the target projection polygon corresponding to the original image, wherein the target projection polygon is the target polygon.

3. The method according to claim 1, characterized in that The target elevation value is the minimum elevation value.

4. The method according to claim 1, characterized in that: The step of generating, for each original image, a first image set of the original image according to the original image, a target projection polygon corresponding to the original image, and the DSM includes: According to the posture information corresponding to the original image, a projection matrix corresponding to the original image is obtained; The color value of each pixel in the first initial orthoimage is calculated based on the projection matrix, the spatial position of the grid points in the target projection polygon corresponding to the original image, and the original image.

5. The method according to any one of claims 1 to 4, characterized in that: The first image set also includes a first initial weight image, The step of generating, for each original image, a first image set of the original image according to the original image, a target projection polygon corresponding to the original image, and the DSM includes: Calculate the pixel value of each pixel in the first initial weighted image corresponding to the original image according to the distance between the pixel corresponding to each grid point in the target projection polygon corresponding to the original image and the central pixel of the original image; The step of generating a target orthophoto of the target scene according to the first initial orthophoto images in each first image set includes: The target orthoimage is generated according to the first initial orthoimage and the first initial weighted image in each first image set.

6. The method according to claim 5, characterized in that The step of generating the target orthophoto according to the first initial orthophoto image and the first initial weighted image in each first image set comprises: For each first initial orthoimage, establishing a first Laplacian pyramid corresponding to the first initial orthoimage; For each first initial weight image, establishing a first Gaussian pyramid corresponding to the first initial weight image; According to the images of each layer in the first Gaussian pyramid, performing layered fusion on the images of each layer in the established first Laplacian pyramid to obtain layered fusion results, wherein the number of the layered fusion results is the same as the number of image layers in the first Laplacian pyramid; The obtained layered fusion results are merged to obtain the target orthophoto.

7. An orthophoto generating device, characterized in that: The device comprises: The acquisition module is used to obtain the original image set and digital surface model DSM of the target scene; A projection module, configured to obtain a target projection polygon corresponding to each original image on the DSM by projecting each original image in the original image set; A processing module, for generating, for each original image, a first image set of the original image according to the original image, a target projection polygon corresponding to the original image and the DSM, wherein the first image set includes a first initial orthoimage corresponding to the original image, and a color value in the first initial orthoimage is calculated by the spatial position of a grid point of the DSM located within the target projection polygon and the original image; The processing module is further used to generate a target orthophoto of the target scene according to the first initial orthophoto images in each first image set; Among them, the processing module is specifically used to: determine the target elevation plane according to the elevation value in the DSM, wherein the target elevation value of the target elevation plane is less than the maximum elevation value in the DSM and not less than the minimum elevation value in the DSM; according to the posture information corresponding to each original image, project each original image onto the target elevation plane to obtain the projection polygon corresponding to each original image, and according to the projection polygon corresponding to each original image, obtain the target projection polygon corresponding to the original image.

8. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the orthophoto generation method described in any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the orthophoto generation method as described in any one of claims 1 to 6 is implemented.

Images