Urban image construction method, device and equipment based on simulated aerial photography, storage medium and program product

Abstract

The invention discloses a city image construction method and device based on simulated aerial photography, equipment, a storage medium and a program product, and relates to the technical field of image processing.The city image construction method based on simulated aerial photography is characterized in that a virtual flight path is generated, and discrete virtual imaging viewpoints are obtained through sampling in the virtual flight path; the whole process of planning-flight-shooting in the physical world is mapped to the virtual environment, so that the simulation of unmanned aerial vehicle teaching, high-risk areas and high-risk environments is not constrained by time and meteorological conditions, and the task planning and verification efficiency of complex aerial photography tasks is improved. A time sequence image is obtained by constructing a virtual imaging curved surface and performing mapping calibration on a time sequence viewpoint image, and a unified mapping projection reference is provided for all discrete time sequence viewpoint images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to methods, apparatus, equipment, storage media, and program products for constructing urban images based on simulated aerial photography. Background Technology

[0002] With the continuous development of drone aerial photography technology, drone aerial surveying has become a mainstream trend in fields such as film and television production and urban planning, replacing manual surveying. Currently, schools are also placing increasing emphasis on teaching the application of drone aerial photography.

[0003] However, current drone aerial photography mainly involves teachers leading students in field shooting, which is time-consuming and has high drone procurement costs. Furthermore, improper student operation can easily cause damage. In addition, field drone teaching is significantly affected by environmental and weather conditions. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, equipment, storage medium and program product for constructing urban images based on simulated aerial photography, which aims to solve the technical problems of existing drone aerial photography, which is prone to damage and is greatly affected by external factors when directly shooting on the field.

[0005] To achieve the above objectives, this application proposes a method for constructing urban images based on simulated aerial photography, the method comprising: A virtual flight path is generated based on the user's simulated aerial photography mission, and discretized sampling is performed on the virtual flight path to obtain a virtual imaging viewpoint; Based on the virtual imaging viewpoint, a virtual imaging surface covering the field of view of the virtual flight path is obtained through global integration. Virtual image sampling is performed in a 3D city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time series image corresponding to the virtual imaging viewpoint; The virtual city image of the simulated aerial photography mission is obtained by fusing the time-series images corresponding to the virtual imaging viewpoint.

[0006] In one embodiment, the step of globally integrating the virtual imaging viewpoints to obtain a virtual imaging surface covering the field of view of the virtual flight path includes: The center trajectory axis of the virtual flight path is determined by fitting the virtual imaging viewpoint. A parametric model of the complex surface is constructed based on the central trajectory axis; The virtual imaging viewpoint is mapped to the complex surface parameterization model to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint. Based on the viewpoint parameter coordinates and the normal direction, a virtual imaging surface covering the field of view of the virtual flight path is obtained through global integration.

[0007] In one embodiment, the step of mapping the virtual imaging viewpoint to the complex surface parameterized model to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint includes: Obtain the three-dimensional spatial coordinates and the line-of-sight vector of the virtual imaging viewpoint; The virtual imaging viewpoint is mapped to the complex surface parameterized model based on the three-dimensional spatial coordinates and the line-of-sight vector to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint.

[0008] In one embodiment, the step of performing virtual image sampling in a 3D city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time-series image corresponding to the virtual imaging viewpoint includes: Based on the virtual imaging viewpoint, virtual image sampling is performed in a 3D city model to obtain an initial viewpoint image sequence; Based on the virtual imaging surface, the initial viewpoint image sequence is reprojected and corrected to obtain the time-series image corresponding to the virtual imaging viewpoint.

[0009] In one embodiment, the step of fusing time-series images corresponding to the virtual imaging viewpoint to obtain virtual city images of the simulated aerial photography mission includes: The panoramic virtual viewpoint image is obtained by fusing statistical information of overlapping regions of time-series images corresponding to the virtual imaging viewpoint. Virtual city images of the simulated aerial photography mission are generated based on the panoramic virtual viewpoint images.

[0010] In one embodiment, the step of discretizing sampling along the virtual flight path to obtain a virtual imaging viewpoint includes: The flight parameters of the UAV are obtained based on the simulated aerial photography mission; Based on the number of generated video frames and the flight parameters, discretized sampling is performed to determine the virtual imaging viewpoint on the virtual flight path.

[0011] Furthermore, to achieve the above objectives, this application also proposes a city image construction device based on simulated aerial photography, the city image construction device based on simulated aerial photography comprising: The viewpoint generation module is used to generate a virtual flight path based on the user's simulated aerial photography mission, and to perform discretization sampling on the virtual flight path to obtain a virtual imaging viewpoint. The panoramic management module is used to perform global integration based on the virtual imaging viewpoint to obtain a virtual imaging surface that covers the field of view of the virtual flight path; The image acquisition module is used to perform virtual image sampling in a three-dimensional city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time series image corresponding to the virtual imaging viewpoint; The image generation module is used to obtain virtual city images of the simulated aerial photography mission based on the time-series images corresponding to the virtual imaging viewpoint.

[0012] In addition, to achieve the above objectives, this application also proposes an urban image construction device based on simulated aerial photography. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the urban image construction method based on simulated aerial photography as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the urban image construction method based on simulated aerial photography as described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the urban image construction method based on simulated aerial photography as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This application generates a virtual flight path based on a user-simulated aerial photography mission, and performs discretized sampling on the virtual flight path to obtain virtual imaging viewpoints. It then globally integrates these virtual imaging viewpoints to obtain a virtual imaging surface covering the field of view of the virtual flight path. Based on the virtual imaging viewpoints and the virtual imaging surface, it performs virtual image sampling in a 3D city model to obtain time-series images corresponding to the virtual imaging viewpoints. Finally, it fuses these time-series images to obtain virtual city images simulating the aerial photography mission. Because it generates a virtual flight path and obtains discrete virtual imaging viewpoints through sampling along the virtual flight path, it maps the entire process of "planning-flight-shooting" in the physical world to the virtual environment, ensuring that the output image sequence meets preset requirements. Furthermore, since it simulates the flight path of a drone, it provides drone teaching and simulation of high-risk areas and environments free from the constraints of time and weather conditions, improving the efficiency of mission planning and verification for complex aerial photography missions. By constructing a virtual imaging surface and mapping and calibrating the time-series viewpoint images, it obtains time-series images, providing a unified mapping and projection benchmark for all discrete time-series viewpoint images. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the urban image construction method based on simulated aerial photography provided in this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the urban image construction method based on simulated aerial photography provided in this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the urban image construction method based on simulated aerial photography provided in this application; Figure 4 This is a schematic diagram of the modular structure of the urban image construction device based on simulated aerial photography, as described in an embodiment of this application. Figure 5 This is a schematic diagram of the hardware operating environment involved in the urban image construction method based on simulated aerial photography in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application embodiment is as follows: a virtual flight path is generated based on the user's simulated aerial photography task, and discretized sampling is performed on the virtual flight path to obtain a virtual imaging viewpoint; the virtual imaging viewpoint is globally integrated to obtain a virtual imaging surface covering the field of view of the virtual flight path; virtual image sampling is performed in a three-dimensional city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time series image corresponding to the virtual imaging viewpoint; and a virtual city image of the simulated aerial photography task is obtained based on the time series image corresponding to the virtual imaging viewpoint.

[0023] This application provides a solution that, by constructing images through simulated drone aerial photography, allows users to understand the drone's flight path and control methods beforehand, as well as the generation of simulated images, reducing the risk of drone damage for student users. Furthermore, because the images are generated through simulation, the impact of weather on teaching is reduced, improving the adaptability of drone aerial photography to various scenarios.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a computer or server, or an electronic device or virtual device capable of performing the above functions. The following description uses a city image construction device based on simulated aerial photography (hereinafter referred to as the construction device) as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, the embodiments of this application provide a method for constructing urban images based on simulated aerial photography, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the urban image construction method based on simulated aerial photography provided in this application.

[0026] In this embodiment, the urban image construction method based on simulated aerial photography includes steps S10 to S40: Step S10: Generate a virtual flight path based on the user's simulated aerial photography mission, and perform discretization sampling on the virtual flight path to obtain a virtual imaging viewpoint.

[0027] It should be noted that during drone aerial photography, in order to obtain target images of a specific area, users typically perform a series of controls on the physical drone and its onboard camera. The combination of these control actions can be considered a drone aerial photography task. In this embodiment, the simulated aerial photography task applies the aforementioned drone aerial photography task to a virtual drone model.

[0028] In some embodiments of this application, the embodiments of this application and the following embodiments use the construction of urban imagery as an example to illustrate the solution of this application. In practical applications, a virtual drone model can be first placed in an initial position in a three-dimensional city model, and then the above-mentioned simulated aerial photography task can be performed through the drone model to generate a flight path of the drone model, which is the virtual flight path. Through the virtual flight path, parameters such as the drone's flight path, flight speed, motion turning, and turning information can be represented.

[0029] It is understood that the initial position mentioned above can be any position selected by the user in the three-dimensional city model, and this application embodiment does not limit this.

[0030] It should be noted that the aforementioned virtual imaging viewpoint can be a momentary, static virtual camera state obtained by discretizing and sampling the virtual flight path, which can be used to characterize the complete imaging parameters of the corresponding point of the physical UAV on the flight path. Specifically, the virtual imaging viewpoint can be a simulation of the imaging parameters of the corresponding position of the physical UAV in the physical world, equivalent to a digital snapshot of the physical UAV in the virtual world.

[0031] It is understood that the above-mentioned complete imaging parameters may include: camera intrinsic parameters, camera extrinsic parameters, imaging attitude angle, imaging relative time, etc. of the imaging unit mounted on the physical UAV, and the embodiments of this application do not limit this.

[0032] It should be noted that when discretizing the virtual flight path, it can be achieved by sampling at a fixed spatial interval or a fixed time interval, such as setting a virtual imaging viewpoint every 10 meters or every 0.03 seconds. This application embodiment does not limit this.

[0033] In some embodiments of this application, when using a fixed time interval, the time interval can be set according to the video frame rate (i.e., the number of generated video frames) corresponding to the generated city image. For example, when the video frame rate of the generated city image is 30 frames, a virtual imaging viewpoint for sampling can be set on the path every 1 / 30 second; when the video frame rate of the generated city image is 60 frames, a virtual imaging viewpoint for sampling can be set on the virtual flight path every 1 / 60 second. That is, the step of discretizing sampling on the virtual flight path to obtain the virtual imaging viewpoint includes: obtaining the flight parameters of the UAV based on the simulated aerial photography task; and performing discretizing sampling based on the number of generated video frames and the flight parameters to determine the virtual imaging viewpoint on the virtual flight path.

[0034] It should be noted that the flight parameters of the aforementioned drone may include the drone's flight direction and speed along the flight path. Based on the drone's flight parameters and the number of video frames, a virtual imaging viewpoint can be determined on the virtual flight path.

[0035] In its specific implementation, the construction device of this application embodiment can generate a virtual flight path corresponding to the physical drone flight based on the user's simulated aerial photography task. It then performs discretized sampling on the virtual flight path according to the drone's flight parameters and the number of video frames generated from the city images to determine the virtual imaging viewpoints on the virtual flight path. By mapping the entire process of "planning-flight-shooting" in the physical world to the virtual environment, the output image sequence can meet preset requirements. Simultaneously, since the shooting is performed by simulating the drone's flight path, it provides drone teaching and simulation of high-risk areas and high-risk environments without being constrained by time and weather conditions, improving the efficiency of task planning and verification for complex aerial photography tasks.

[0036] Step S20: Perform global integration based on the virtual imaging viewpoint to obtain a virtual imaging surface that covers the field of view of the virtual flight path; Step S30: Based on the virtual imaging viewpoint and the virtual imaging surface, perform virtual image sampling in the three-dimensional city model to obtain the time series image corresponding to the virtual imaging viewpoint; Step S40: Based on the time series images corresponding to the virtual imaging viewpoint, perform fusion to obtain the virtual city image of the simulated aerial photography mission.

[0037] It should be noted that the aforementioned virtual imaging surface can be a continuous parametric surface model constructed by globally integrating the position information and line-of-sight vectors of all virtual imaging viewpoints. By constructing a continuous imaging virtual surface that can cover the entire flight observation range, a unified framework can be provided for the image projection, correction, and stitching of virtual images generated from various virtual imaging viewpoints. This allows initial viewpoint image sequences acquired from different viewpoints to be accurately aligned and fused within the same reference frame, thereby generating seamless panoramic virtual viewpoint images.

[0038] It is understandable that the line-of-sight vector of a virtual imaging viewpoint can represent the shooting direction of the UAV at that virtual imaging viewpoint. By rendering images based on a 3D city model at each virtual imaging viewpoint, the initial viewpoint image corresponding to each virtual imaging viewpoint can be determined. Since each virtual imaging viewpoint can be regarded as a position at different times on the UAV's flight path, it can be assumed that there is a temporal relationship between the initial viewpoint images of these virtual imaging viewpoints. Based on the position / sequence of the virtual imaging viewpoints and the UAV's flight parameters, corresponding temporal labels are set to obtain temporal viewpoint images. The images obtained by mapping these temporal viewpoint images onto the virtual imaging surface are time-series images.

[0039] In its specific implementation, the construction device of this application embodiment can globally integrate the position and viewpoint direction of the virtual imaging viewpoint to determine a virtual imaging surface covering the entire virtual flight path's field of view. By mapping and calibrating the temporal viewpoints acquired and rendered by each virtual imaging viewpoint using this virtual imaging surface, the corresponding time-series image for each virtual imaging viewpoint can be determined. Since the time-series image is obtained by constructing a virtual imaging surface and mapping and calibrating the temporal viewpoint image, a unified mapping and projection reference is provided for all discrete temporal viewpoint images, avoiding the defect of traditional multi-viewpoint stitching errors accumulating frame by frame. By fusing these time-series images, a virtual city image simulating an aerial photography mission can be obtained.

[0040] This application embodiment generates a virtual flight path based on a user-simulated aerial photography mission, and performs discretized sampling on the virtual flight path to obtain virtual imaging viewpoints. Global integration is then performed based on the virtual imaging viewpoints to obtain a virtual imaging surface covering the field of view of the virtual flight path. Virtual image sampling is then performed on a 3D city model based on the virtual imaging viewpoints and the virtual imaging surface to obtain time-series images corresponding to the virtual imaging viewpoints. Finally, the time-series images corresponding to the virtual imaging viewpoints are fused to obtain virtual city images simulating the aerial photography mission. Because a virtual flight path is generated and discrete virtual imaging viewpoints are obtained through sampling along the virtual flight path, the entire process of "planning-flight-shooting" in the physical world is mapped to the virtual environment, ensuring that the output image sequence meets preset requirements. Furthermore, since the shooting is performed by simulating the flight path of a drone, it provides drone teaching, simulation of high-risk areas, and high-risk environments without being constrained by time and weather conditions, improving the efficiency of mission planning and verification for complex aerial photography missions. By constructing a virtual imaging surface and mapping and calibrating the time-series viewpoint images to obtain time-series images, a unified mapping projection benchmark is provided for all discrete time-series viewpoint images.

[0041] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating Embodiment 2 of the urban image construction method based on simulated aerial photography provided in this application.

[0042] like Figure 2 As shown in the embodiment of this application, the step of performing global integration based on the virtual imaging viewpoint to obtain a virtual imaging surface covering the field of view of the virtual flight path includes: Step S21: Based on the virtual imaging viewpoint, fit the data to determine the center trajectory axis of the virtual flight path; Step S22: Construct a parametric model of the complex surface based on the central trajectory axis.

[0043] It should be noted that the aforementioned central trajectory axis can be a reference spatial curve determined by path smoothing and fitting based on the virtual flight path, used to construct a parametric model of the complex surface. By using the continuously differentiable central trajectory axis as the reference spatial axis, the parametric model of the complex surface can be determined using the radial distance function. Specifically, the original virtual flight path consists of a series of points and may contain some high-frequency jitter. By smoothing and filtering or curve fitting the virtual flight path, flight control noise and minor fluctuations can be eliminated, thus obtaining the central trajectory axis. Based on this central trajectory axis, the parametric model of the complex surface can be constructed.

[0044] It should be explained that the above parametric model of the complex surface can be a parametric representation of the complex surface (i.e., the virtual imaging surface). Points on the parametric model of the complex surface can be represented as follows: ; in, The arc length on the central trajectory axis is... point The corresponding position vector, This represents the position vector of a point on the parametric model of the complex surface; Indicates at point A direction vector within the normal plane. and The angle between them The range of the value is [0, 2π). . The radius function is used to represent a point on the central trajectory axis. At that location, along the direction vector The direction, from point Points on the parametric model of the complex surface The corresponding distance.

[0045] It should be noted that for points on the central trajectory axis A Frenet frame can be defined, which can consist of three mutually orthogonal unit vector fields. Let Frénet's frame have a unit principal normal field, the direction of which is a point on the central trajectory axis. The point is located in the direction pointing towards the center of curvature, with a length equal to the axis of the central trajectory. The curvature at that point. For the Frenier frame, the unit tangent vector field is specifically the axis of the central trajectory at point [point missing]. The unit tangential vector at that location. Denotes the unit binormal vector field of the Frenet frame. , and , It constitutes a set of orthogonal unit bases in space.

[0046] In practical applications, drone aerial photography or simulated aerial photography can be viewed as a series of localized images with limited viewing angles, acquired at different times and spatial locations (i.e., discrete virtual imaging viewpoints). This application's embodiments utilize virtual imaging surfaces to map any point in the time-series image rendered from the virtual imaging viewpoint onto a unified complex surface, avoiding ghosting and double images caused by mismatched projection planes and accumulated image alignment errors in traditional stitching methods.

[0047] Step S23: Map the virtual imaging viewpoint to the complex surface parameterization model to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint; Step S24: Based on the viewpoint parameter coordinates and the normal direction, perform global integration to obtain a virtual imaging surface that covers the field of view of the virtual flight path.

[0048] It is understandable that, based on the conversion formula between points on the parametric model of the complex surface and any point on the central trajectory axis, it is possible to map the virtual imaging viewpoint to the parametric model of the complex surface.

[0049] It should be noted that the three-dimensional spatial coordinates and line-of-sight vector of each virtual imaging viewpoint can be determined. The three-dimensional spatial coordinates refer to the coordinate position of the virtual imaging viewpoint in the three-dimensional city model, and the line-of-sight vector can be understood as the attitude angle of the drone in the air, specifically representing the direction in which the drone takes pictures. Based on the three-dimensional spatial coordinates and line-of-sight vectors of each virtual imaging viewpoint, these discrete virtual imaging viewpoints can be mapped to a complex surface parameterized model to obtain the corresponding viewpoint parameter coordinates and normal direction. That is, the step of mapping the virtual imaging viewpoint to the complex surface parameterized model to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint includes: obtaining the three-dimensional spatial coordinates and line-of-sight vector of the virtual imaging viewpoint; and mapping the virtual imaging viewpoint to the complex surface parameterized model based on the three-dimensional spatial coordinates and the line-of-sight vector to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint.

[0050] In some embodiments of this application, for a virtual imaging viewpoint Based on its three-dimensional spatial coordinates, the vertical projection point of the virtual imaging viewpoint on the central trajectory axis can be determined to minimize the distance from the virtual imaging viewpoint to the central trajectory axis. When determining the vertical projection point, the arc length corresponding to that point can be determined; this arc length parameter is the axial parameter corresponding to the virtual imaging viewpoint. Based on the three-dimensional spatial coordinates of the virtual imaging viewpoint, its corresponding vertical projection point, and the axial parameter, the azimuth angle and axial distance of the virtual imaging viewpoint can be determined. This axial distance is the length of the line segment connecting the virtual imaging viewpoint and the vertical projection point, and the azimuth angle can be determined by establishing a Freney frame based on the vertical projection point. Specifically, a Freney frame can be established based on the vertical projection point, and the azimuth angle is the angle between the vector formed by the virtual imaging viewpoint and the vertical projection point and the normal plane (the plane composed of the unit principal normal field and the unit secondary normal field). Through the above operations, any point can be converted into a set of parameters. . This represents the arc length of the perpendicular projection point corresponding to that point. It indicates the azimuth angle around the axis (which side of the central trajectory axis the point is on). It represents the axial distance (the distance of a point from the axis of the central trajectory). It indicates the direction of the desired surface normal (opposite to the direction of the line of sight vector, indicating which direction the desired complex surface should face).

[0051] In some embodiments of this application, the azimuth angle and axial distance corresponding to the virtual imaging viewpoint can be found on a complex surface. Specifically, this is achieved through the aforementioned parameter sets corresponding to each virtual imaging viewpoint. It can realize the radial distance function Fitting is performed to make the radial distance function The function value should be as close as possible to the distance. And the actual normal direction of the complex surface at that point should be as close as possible to the normal direction of the desired surface. This allows arbitrary points to be mapped onto a virtual imaging surface based on a complex surface parameterization model.

[0052] This application embodiment determines the central trajectory axis of the virtual flight path by fitting based on the virtual imaging viewpoint; constructs a complex surface parameterized model based on the central trajectory axis; maps the virtual imaging viewpoint to the complex surface parameterized model to obtain the viewpoint parameter coordinates and normal direction of the virtual imaging viewpoint; and globally integrates the viewpoint parameter coordinates and normal direction to obtain a virtual imaging surface covering the field of view of the virtual flight path. By using discrete virtual imaging viewpoints for curve fitting to determine the central trajectory axis, noise in the viewpoint data is reduced, and sampling errors are decreased. By constructing a complex surface parameterized model, the problem of constructing a three-dimensional surface is transformed into constructing a surface in a two-dimensional parameter domain. Determine a radial distance function. This addresses the problem by reducing computational dimensionality and complexity.

[0053] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and / or second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating Embodiment 3 of the urban image construction method based on simulated aerial photography provided in this application.

[0054] like Figure 3 As shown in the embodiment of this application, the step of performing virtual image sampling in a 3D city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time-series image corresponding to the virtual imaging viewpoint includes: Step S31: Based on the virtual imaging viewpoint, perform virtual image sampling in the three-dimensional city model to obtain an initial viewpoint image sequence; Step S32: Based on the virtual imaging surface, perform reprojection correction on the initial viewpoint image sequence to obtain the time-series image corresponding to the virtual imaging viewpoint.

[0055] It should be noted that for each virtual imaging viewpoint, an initial viewpoint image sequence can be obtained by virtual image sampling in the 3D city model. By reprojecting and correcting these initial viewpoint image sequences onto the virtual imaging surface, the time series image corresponding to the virtual imaging viewpoint can be obtained.

[0056] In some embodiments of this application, the step of fusing the time-series images corresponding to the virtual imaging viewpoint to obtain the virtual city image of the simulated aerial photography mission includes: fusing the overlapping area statistical information of the time-series images corresponding to the virtual imaging viewpoint to obtain a panoramic virtual viewpoint image; and generating the virtual city image of the simulated aerial photography mission based on the panoramic virtual viewpoint image.

[0057] It should be noted that for time-series images projected onto a virtual imaging surface, the fields of view of multiple adjacent virtual imaging viewpoints overlap on the virtual imaging surface. By statistically analyzing this overlap information, the time-series images corresponding to adjacent virtual imaging viewpoints can be fused to obtain a panoramic virtual viewpoint image. During fusion, consistency enhancement, edge preservation, and smooth transitions can be used to improve the effect of the panoramic viewpoint image. Based on the final generated panoramic virtual viewpoint image, segmentation can be performed based on task parameters such as video generation frame rate, resolution, and duration to generate dynamic virtual city images. This application embodiment does not limit the specific methods of image fusion and image generation described above; they can be set according to the needs of actual applications.

[0058] This application embodiment obtains an initial viewpoint image sequence by performing virtual image sampling in a 3D city model based on a virtual imaging viewpoint; it then performs reprojection correction on the initial viewpoint image sequence based on a virtual imaging surface to obtain a time-series image corresponding to the virtual imaging viewpoint. The overlapping region statistics of the time-series images corresponding to the virtual imaging viewpoint are fused to obtain a panoramic virtual viewpoint image; and a virtual city image simulating an aerial photography mission is generated based on the panoramic virtual viewpoint image. Because the initial viewpoint image sequence is reprojected, distortions in the original image are eliminated; and by fusing time-series images from different virtual imaging viewpoints, the consistency of the generated image is improved, thus enhancing the quality of the generated image.

[0059] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the urban image construction method based on simulated aerial photography in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0060] This application also provides a device for constructing urban images based on simulated aerial photography; please refer to... Figure 4 , Figure 4 This is a schematic diagram of the module structure of the urban image construction device based on simulated aerial photography according to an embodiment of this application. The urban image construction device based on simulated aerial photography includes: The viewpoint generation module 10 is used to generate a virtual flight path based on the user's simulated aerial photography mission, and to perform discretization sampling on the virtual flight path to obtain a virtual imaging viewpoint. The panoramic management module 20 is used to perform global integration based on the virtual imaging viewpoint to obtain a virtual imaging surface that covers the field of view of the virtual flight path. Image acquisition module 30 is used to perform virtual image sampling in a three-dimensional city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time series image corresponding to the virtual imaging viewpoint; The image generation module 40 is used to obtain virtual city images of the simulated aerial photography mission based on the time series images corresponding to the virtual imaging viewpoint.

[0061] The urban image construction device based on simulated aerial photography provided in this application, employing the urban image construction method based on simulated aerial photography in the above embodiments, can solve the technical problems of existing drone aerial photography, which is prone to damage and highly susceptible to external influences when directly conducting on-site shooting. Compared with the prior art, the beneficial effects of the urban image construction device based on simulated aerial photography provided in this application are the same as those of the urban image construction method based on simulated aerial photography provided in the above embodiments, and other technical features in the urban image construction device based on simulated aerial photography are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0062] This application provides a city image construction device based on simulated aerial photography. The city image construction device based on simulated aerial photography includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the city image construction method based on simulated aerial photography in the above embodiment 1.

[0063] The following is for reference. Figure 5This document illustrates a structural diagram of a city image construction device based on simulated aerial photography, suitable for implementing embodiments of this application. The city image construction device based on simulated aerial photography in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The urban image construction device based on simulated aerial photography shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0064] like Figure 5 As shown, the city image construction device based on simulated aerial photography may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the city image construction device based on simulated aerial photography. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the city image building equipment based on simulated aerial photography to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a city image building equipment based on simulated aerial photography with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0065] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0066] The urban image construction device based on simulated aerial photography provided in this application, employing the urban image construction method based on simulated aerial photography in the above embodiments, can solve the technical problems of existing drone aerial photography directly shooting on-site, which is prone to damage and is greatly affected by external factors. Compared with the prior art, the beneficial effects of the urban image construction device based on simulated aerial photography provided in this application are the same as those of the urban image construction method based on simulated aerial photography provided in the above embodiments, and other technical features of the urban image construction device based on simulated aerial photography are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0067] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0068] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0069] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the urban image construction method based on simulated aerial photography in the above embodiments.

[0070] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0071] The aforementioned computer-readable storage medium may be included in a simulated aerial photography-based urban image construction device; or it may exist independently and not be assembled into a simulated aerial photography-based urban image construction device.

[0072] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the simulated aerial photography-based urban image construction device, cause the simulated aerial photography-based urban image construction device to: A virtual flight path is generated based on the user's simulated aerial photography mission, and discretized sampling is performed on the virtual flight path to obtain a virtual imaging viewpoint; Based on the virtual imaging viewpoint, a virtual imaging surface covering the field of view of the virtual flight path is obtained through global integration. Virtual image sampling is performed in a 3D city model based on the virtual imaging viewpoint and the virtual imaging surface to obtain a time series image corresponding to the virtual imaging viewpoint; The virtual city image of the simulated aerial photography mission is obtained by fusing the time-series images corresponding to the virtual imaging viewpoint.

[0073] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0076] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described urban image construction method based on simulated aerial photography. This solves the technical problem that existing drone aerial photography, which directly involves on-site field shooting, is prone to damage and is highly susceptible to external influences. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the urban image construction method based on simulated aerial photography provided in the above embodiments, and will not be repeated here.

[0077] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the urban image construction method based on simulated aerial photography as described above.

[0078] The computer program product provided in this application can solve the technical problems of existing drone aerial photography, which is prone to damage and is highly susceptible to external influences when directly shooting on-site. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the urban image construction method based on simulated aerial photography provided in the above embodiments, and will not be repeated here.

[0079] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for constructing city images based on simulated aerial photography, characterized in that, The method comprises: generating a virtual flight path based on a user's simulated aerial photography task and discretely sampling on the virtual flight path to obtain virtual imaging viewpoints; globally integrating according to the virtual imaging viewpoints to obtain a virtual imaging surface covering a field of view range of the virtual flight path; performing virtual image sampling in a three-dimensional city model based on the virtual imaging viewpoints and the virtual imaging surface to obtain time sequence images corresponding to the virtual imaging viewpoints; fusing the time sequence images corresponding to the virtual imaging viewpoints to obtain virtual city images of the simulated aerial photography task.

2. The method of claim 1, wherein the analog aerial photo-based city image construction method is characterized by, The step of globally integrating according to the virtual imaging viewpoints to obtain a virtual imaging surface covering a field of view range of the virtual flight path comprises: fitting based on the virtual imaging viewpoints to determine a central trajectory axis of the virtual flight path; constructing a complex surface parameterization model based on the central trajectory axis; mapping the virtual imaging viewpoints to the complex surface parameterization model to obtain viewpoint parameter coordinates and normal directions of the virtual imaging viewpoints; globally integrating based on the viewpoint parameter coordinates and the normal directions to obtain a virtual imaging surface covering a field of view range of the virtual flight path.

3. The method of claim 2, wherein the analog aerial photo-based city image construction method is characterized by, The step of mapping the virtual imaging viewpoints to the complex surface parameterization model to obtain viewpoint parameter coordinates and normal directions of the virtual imaging viewpoints comprises: obtaining three-dimensional space coordinates and a line of sight direction vector of the virtual imaging viewpoints; mapping the virtual imaging viewpoints to the complex surface parameterization model according to the three-dimensional space coordinates and the line of sight direction vector to obtain viewpoint parameter coordinates and normal directions of the virtual imaging viewpoints.

4. The method of claim 1, wherein the analog aerial photo-based city image construction method is characterized by, The step of performing virtual image sampling in a three-dimensional city model based on the virtual imaging viewpoints and the virtual imaging surface to obtain time sequence images corresponding to the virtual imaging viewpoints comprises: performing virtual image sampling in a three-dimensional city model based on the virtual imaging viewpoints to obtain an initial viewpoint image sequence; correcting the initial viewpoint image sequence by re-projection based on the virtual imaging surface to obtain time sequence images corresponding to the virtual imaging viewpoints.

5. The method of claim 1, wherein the analog aerial photo-based city image construction method is characterized by, The step of fusing the time sequence images corresponding to the virtual imaging viewpoints to obtain virtual city images of the simulated aerial photography task comprises: fusing overlapping area statistical information of the time sequence images corresponding to the virtual imaging viewpoints to obtain a panoramic virtual viewpoint image; generating virtual city images of the simulated aerial photography task based on the panoramic virtual viewpoint image.

6. The method of claim 1, wherein the analog aerial photo-based city image construction method is characterized by, The step of discretely sampling on the virtual flight path to obtain virtual imaging viewpoints comprises: obtaining flight parameters of a UAV based on the simulated aerial photography task; discretely sampling based on the number of generated video frames and the flight parameters to determine virtual imaging viewpoints on the virtual flight path.

7. A device for constructing urban images based on simulated aerial photography, characterized in that, The device for constructing city images based on simulated aerial photography comprises: a viewpoint generation module configured to generate a virtual flight path based on a user's simulated aerial photography task and discretely sample on the virtual flight path to obtain virtual imaging viewpoints; The panoramic management module is configured to globally integrate the virtual imaging viewpoints to obtain a virtual imaging curved surface covering a field of view range of the virtual flight path; The image acquisition module is configured to perform virtual image sampling in the three-dimensional city model based on the virtual imaging viewpoints and the virtual imaging curved surface to obtain time-series images corresponding to the virtual imaging viewpoints; The image generation module is configured to obtain virtual city images of the simulated aerial photography task based on the time-series images corresponding to the virtual imaging viewpoints.

8. A city image construction device based on simulated aerial photography, characterized by comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the city image construction method based on simulated aerial photography according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the city image construction method based on simulated aerial photography according to any one of claims 1 to 6.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the city image construction method based on simulated aerial photography according to any one of claims 1 to 6.

Images