Image occlusion detection method and device

Abstract

The present application discloses an image occlusion detection method and device. The method includes: calculating the ground coverage of the original image according to the original image and its corresponding RPC data; constructing a TIN model according to the ground coverage and its corresponding DHM data; constructing an occlusion mask image according to the ground coverage, and the pixel value of each pixel in the occlusion mask image is a first value; for any i-th pixel in the occlusion mask image, calculating the photographic light of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image; using a ray tracing algorithm, according to the photographic light of the i-th pixel and the TIN model, obtaining the occluded pixel in the occlusion mask image, and setting the value of the occluded pixel to a second value. In this way, the occluded pixel in the occlusion mask image can be obtained, and the pixel whose pixel value is set to the second value corresponds to the pixel point in the original image, which is the occluded point.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to an image occlusion detection method and device. Background Art

[0002] In the process of acquiring satellite remote sensing images, the imaging light is blocked by the high-altitude ground objects, resulting in the appearance of shielded areas in the remote sensing images. In the process of indirect orthorectification, the grayscale values ​​of these shielded areas that are not detected by remote sensing will be filled with the grayscale values ​​of other ground objects that block the area, thus forming pseudo targets in the areas that should be blank in the final correction result image, affecting the thematic analysis of satellite images. Therefore, it is necessary to accurately detect and identify the shielded areas.

[0003] Existing occlusion detection methods are generally applicable to remote sensing images that adopt strict geometric models. However, since high-resolution satellites generally use the rational polynomial coefficients (RPC) model as a positioning model, it cannot provide information such as exterior orientation elements, which makes the existing occlusion detection algorithms no longer applicable.

[0004] It is hoped that a new technical solution can be developed to realize the image occlusion detection method based on the RPC model. Summary of the invention

[0005] The present application discloses an image occlusion detection method and device.

[0006] In a first aspect, a method for detecting image occlusion is provided, the method comprising:

[0007] Calculate the ground coverage of the original image based on the original image and its corresponding RPC data;

[0008] Constructing a triangulated irregular network (TIN) model according to the ground coverage and its corresponding digital height model (DHM) data;

[0009] Constructing an occlusion mask image according to the ground coverage range, wherein the pixel value of each pixel in the occlusion mask image is a first value;

[0010] For any i-th pixel in the occlusion mask image, calculating a photographic light vector of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image;

[0011] By using a ray tracing algorithm, according to the photographic ray vector of the i-th pixel and the TIN model, the obscured pixel in the occlusion mask image is obtained, and the value of the obscured pixel is set to the second value.

[0012] In a possible implementation, calculating the photographic light vector of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image includes:

[0013] The photographic light vector of the i-th pixel is calculated according to the maximum value and the minimum value in the DHM data and the coordinates corresponding to the i-th pixel in the original image.

[0014] In a possible implementation, the obtaining the obscured pixel in the occlusion mask image according to the photographic ray vector of the i-th pixel and the TIN model using a ray tracing algorithm includes:

[0015] Dividing the TIN model into a plurality of TIN blocks according to the photographic light vectors of the corner pixels and the DHM data; the corner pixels are pixels in the occlusion mask image corresponding to the vertices of the original image;

[0016] Obtaining a TIN block corresponding to the i-th pixel from the TIN model according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel;

[0017] By using a ray tracing algorithm, the obscured pixel in the occlusion mask image is obtained according to the photographic ray vector of the i-th pixel and the TIN block corresponding to the i-th pixel.

[0018] In a possible implementation, dividing the TIN model into a plurality of TIN blocks according to the photographic light vector of the corner pixel and the DHM data includes:

[0019] Calculate the maximum azimuth angle difference and the average azimuth angle of the photographic light rays of the original image vertices according to the photographic light ray vectors of the corner pixel and the DHM data;

[0020] Calculating the block width of the TIN model according to the maximum azimuth difference and the maximum ray length of the ground coverage range;

[0021] The TIN model is divided into a plurality of TIN blocks according to the block width of the TIN model and the photographic light direction corresponding to the average azimuth angle.

[0022] In a possible implementation, dividing the TIN model into a plurality of TIN blocks according to the block width of the TIN model and the photographic light direction corresponding to the average azimuth angle includes:

[0023] Constructing a circumscribed rectangle of the ground coverage area according to the direction of the photographic light corresponding to the average azimuth angle;

[0024] According to the block width of the TIN model, the circumscribed rectangular block is divided into a plurality of circumscribed rectangular blocks along the perpendicular direction of the photographic light corresponding to the average azimuth angle;

[0025] The TIN model is divided into TIN blocks according to the overlapping area between the circumscribed rectangular block and the ground coverage area.

[0026] In a possible implementation, obtaining a TIN block corresponding to the i-th pixel from the TIN model according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel includes:

[0027] According to the coordinates of the i-th pixel in the occlusion mask image and the DHM data, and in accordance with the principle of being closest to the center line, a TIN block corresponding to the i-th pixel is selected from the TIN model.

[0028] In a possible implementation, obtaining the occluded pixel in the occlusion mask image according to the photographic light vector of the i-th pixel and the TIN block corresponding to the i-th pixel includes:

[0029] From the TIN block corresponding to the i-th pixel, intercept the triangle within the projection range of the photographic light of the i-th pixel to obtain a triangle set;

[0030] According to the triangle set and the photographic ray vector of the i-th pixel, a ray tracing algorithm is used to obtain the occluded pixel in the occlusion mask image.

[0031] In a second aspect, an image occlusion detection device is provided, the device comprising:

[0032] A first calculation unit, configured to calculate the ground coverage of the original image according to the original image and its corresponding RPC data;

[0033] A model building unit, used for building a TIN model according to the ground coverage and its corresponding DHM data;

[0034] An image construction unit, configured to construct an occlusion mask image according to the ground coverage, wherein the value of each pixel in the occlusion mask image is a first value;

[0035] a second calculation unit, for calculating, for any i-th pixel in the occlusion mask image, a photographic light vector of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image;

[0036] The occlusion judgment unit is used to obtain the occluded pixel in the occlusion mask image according to the photographic light vector of the i-th pixel and the TIN model by using a ray tracing algorithm, and set the value of the occluded pixel to a second value.

[0037] In a third aspect, a computing device is provided, wherein the computing device comprises a processor, and when the processor executes computer instructions / codes stored in a memory, the method according to any one of the first aspects is implemented.

[0038] According to a fourth aspect, a computer-readable storage medium is provided, on which computer instructions / codes are stored. When the computer instructions / codes are executed in a computing device, the computing device executes the method described in any one of the first aspects.

[0039] Through the technical solution provided in the embodiment of the present application, the ground coverage of the original image can be calculated through the original image and its corresponding RPC data; then, a TIN model and an occlusion mask image are constructed according to the ground coverage and its corresponding DHM data, and the value of each pixel in the occlusion mask image is set to a first value; then, for any i-th pixel in the occlusion mask image, a photographic ray vector suitable for the RPC model can be calculated according to the DHM data and the coordinates of the i-th pixel corresponding to the original image; further, a ray tracing algorithm can be used to obtain the obscured pixel in the occlusion mask image according to the photographic ray vector of the i-th pixel and the TIN model, and the pixel value of the obscured pixel is set to a second value, and the pixel point corresponding to the pixel whose pixel value is set to the second value in the original image is the obscured point. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of an image occlusion detection method provided in an embodiment of the present application;

[0042] Figure 2A schematic diagram of a TIN model structure provided in an embodiment of the present application;

[0043] Figure 3 for Figure 1 A flowchart of a specific implementation of step 107;

[0044] Figure 4 for Figure 3 A flowchart of a specific implementation of step 301;

[0045] Figure 5 A schematic diagram of a process for calculating the maximum azimuth angle difference of photographic light provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of a process of dividing a TIN model into blocks provided in an embodiment of the present application;

[0047] Figure 7 An occlusion mask binary image provided by an embodiment of the present application;

[0048] Figure 8 A schematic diagram of the structure of an image occlusion detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0050] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions, and any embodiment or design described as "exemplary" should not be construed as being preferred or advantageous over other embodiments or designs.

[0051] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "plurality" means two or more.

[0052] In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0053] In remote sensing images, the RPC model is a mathematical model used to describe the geometric positioning of remote sensing images. RPC data uses polynomial functions to describe the relationship between pixel coordinates and geographic coordinates, thereby achieving geometric correction and positioning of remote sensing images. RPC data usually consists of two rational polynomial functions, one for describing the relationship between pixel coordinates and geographic coordinates in the row direction; the other for describing the relationship between pixel coordinates and geographic coordinates in the column direction. These two rational polynomial functions are called row rational polynomials and column rational polynomials, respectively. Through RPC data, the pixel coordinates in remote sensing images can be mapped to the geographic coordinate system to achieve pixel-level geometric positioning and correction, providing accurate geographic reference for subsequent remote sensing data processing and analysis. RPC data can be provided by remote sensing image manufacturers or data providers and stored in image files as part of the image data.

[0054] DHM is a geographic data model that digitally represents the elevation information of each point on the surface. It usually exists in the form of a raster or a point cloud. DHM data is usually constructed from elevation data obtained by laser radar (LiDAR), stereo photography or other elevation measurement technologies. It describes the three-dimensional morphology of the surface, including the height and shape of mountains, rivers, buildings and other objects. DHM data is widely used in geographic information systems (GIS), land planning, environmental monitoring, water resources management and other fields. In these fields, DHM data can be used to create 3D terrain models, conduct terrain analysis, extract terrain features, assist in geological exploration, flood simulation and other tasks.

[0055] Digital Elevation Model (DEM) and Digital Surface Model (DSM) are two common digital height models used to describe the elevation information of the earth's surface, but they differ in data processing and application. DEM data describes the digital elevation model of the terrain, which usually only includes the elevation information of the terrain, without considering the height of objects such as buildings and trees. It is mainly used in terrain analysis, hydrological simulation, geomorphological research and other fields, and the description of terrain features such as terrain undulations, mountains, and rivers is more accurate. DSM data describes the digital surface model of the earth's surface, including the elevation information of all objects on the surface (such as buildings, trees, etc.) and the terrain. DSM not only includes the elevation information of the terrain, but also takes into account the height of the objects. Therefore, it has a wide range of applications in urban construction planning, remote sensing image interpretation, landscape generation and other fields.

[0056] The TIN model is a terrain model used to represent the surface or other geographical phenomena. It consists of a series of irregular triangles, and the vertices of each triangle correspond to the location of a sampling point or other geographical entity on the surface; there is a topological relationship between the triangles in the TIN model, that is, they are connected to each other through shared vertices and edges to form a connected network structure. The TIN model is commonly used in the fields of geographic information systems (GIS), terrain analysis, geological exploration, hydrological simulation, etc.; as well as for applications such as generating 3D terrain models, performing terrain analysis and visualization, and extracting terrain features.

[0057] The Earth-Centered-Earth-Fixed (ECEF) coordinate system is a spatial rectangular coordinate system with its origin located at the center of mass of the Earth and fixed to the Earth. It is used to describe and locate points on the Earth. In the ECEF coordinate system, the center of mass of the Earth is defined as the origin of the coordinate system, and the coordinate axes are aligned with the Earth's rotation axis. The ECEF coordinates of the ground point have a fixed spatial relationship with the Earth and there is no relative motion. The three coordinate axes of the ECEF coordinate system are the X-axis located in the equatorial plane and pointing to the meridian of longitude 0 degrees, that is, the prime meridian (usually the Greenwich meridian); the Y-axis located in the equatorial plane and perpendicular to the X-axis and pointing to the meridian of longitude 90 degrees; and the Z-axis perpendicular to the XY plane and pointing to the North Pole. ECEF coordinates are usually expressed in a rectangular coordinate system, and their units are usually meters. One of its advantages is that it is relatively simple to directly convert between fixed points on the Earth and points on the Earth's surface, so it is widely used in global navigation satellite systems and other space navigation applications.

[0058] A ground coordinate system (such as a geographic coordinate system or a plane rectangular coordinate system) is a coordinate system that uses the surface of the earth as a reference. It usually uses longitude and latitude or the X, Y coordinates of a plane rectangular coordinate system to describe the location of a point on the earth. This coordinate system is often used in mapping, GPS positioning and other fields.

[0059] The obscured area of ​​remote sensing images is caused by the obstruction of imaging light by objects at a certain height. It is common in mountainous areas and urban areas, and is aggravated as the oblique imaging angle of the remote sensing camera increases. In the process of indirect orthorectification, the obscured area will be filled with the gray value of the obstructing object, resulting in repeated mapping. Therefore, detecting and identifying obscured areas is critical for generating high-quality orthophotos, especially for orthophoto processing of remote sensing images with large inclination angles.

[0060] Since high-resolution satellite images generally use RPC as a positioning model and do not provide information such as external orientation elements, the existing occlusion detection algorithms based on strict geometric models are no longer applicable. In addition, the algorithms for determining occlusion relationships mainly include Z-Buffer and ray tracing. Among them, the ray tracing method has high accuracy, but the amount of calculation is large and the execution efficiency is very low. One of the important reasons is that the surface triangulation of irrelevant areas introduces too many invalid calculations. How to improve the efficiency of ray tracing occlusion detection and realize the rapid and accurate identification of occluded areas in high-resolution satellite images is also one of the problems that need to be solved.

[0061] In order to accurately detect the occluded area in the original image, an embodiment of the present specification provides an image occlusion detection method.

[0062] Figure 1 A flow chart showing the image occlusion detection method is shown in FIG. Figure 1 As shown, the method may include but is not limited to part or all of the following steps S101-S109. The method may be executed by any device, platform, equipment or equipment cluster with computing / processing capabilities, such as by Figure 1 The examples are executed on a computing device of the example.

[0063] Step S101, calculating the ground coverage of the original image according to the original image and its corresponding RPC data.

[0064] Reverse projection can be performed based on the RPC data, and the image coordinates of the pixels on the four sides of the original image in the original image are converted into coordinates in the ground coordinate system according to the pixels on the four sides of the original image, so as to obtain the geographic coordinates of the pixels on the four sides. Then, the ground range covered by the original image is calculated based on the geographic coordinates of the pixels on the four sides. For example, the ground coverage range can be determined by calculating the minimum enclosing rectangle of the geographic coordinates of the pixels on the four sides.

[0065] In a possible implementation, the ground coverage of the original image may also be calculated in combination with DHM data.

[0066] Step S103: construct a TIN model according to the ground coverage and its corresponding DHM data.

[0067] By using a triangulation algorithm, a set of connected triangular meshes can be generated based on the location and elevation information of a set of discrete points to cover the entire ground coverage. For example, first, a set of discrete points for constructing a triangular network can be determined based on the location of the ground coverage, and each discrete point has its own elevation information; wherein the discrete points can be sampling points on the ground coverage or grid points in DHM data. Subsequently, based on these discrete points, a triangulation algorithm can be used to construct triangles between the discrete points, and ensure that each discrete point belongs to a triangle, and there is no gap or overlap between the triangles, thereby forming a connected triangular network.

[0068] For example, the coordinates of all grid points in the DHM data can be first converted from the object coordinate system to the ECEF coordinate system, and then the triangulation algorithm can be used to establish the following ECEF coordinate system for each grid point: Figure 2 The TIN model shown.

[0069] Step S105, constructing a mask binary image of the blank space according to the ground coverage, wherein the pixel value of each pixel in the mask binary image is set to the first value; for any i-th pixel in the mask binary image, calculating the photographic light vector of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image. A pixel whose pixel value of the mask binary image is the first value indicates that the pixel is not blocked. For example, the pixel value of each pixel in the mask binary image can be initialized to 0.

[0070] Each pixel in the occlusion mask image can be found in the corresponding position in the ground coverage area and the corresponding pixel in the original image. When the original image is captured, light is reflected from the ground point in the ground coverage area to the sensor and converted into a digital image. This light is the photographic light of the position in the ground coverage area at the corresponding pixel of the original image, that is, the photographic light of the corresponding pixel of the occlusion mask image.

[0071] In a possible implementation, the photographic light vector of the ith pixel may be calculated according to the maximum value, the minimum value in the DHM data, and the coordinates corresponding to the ith pixel in the original image.

[0072] The intersection points of the photographic light of the ith pixel and the maximum elevation surface and the minimum elevation surface in the ground coverage range can be used as two calculation points for calculating the photographic light vector of the ith pixel. max , the minimum value is Z min , the ground coordinates of the two calculation points corresponding to the ground coverage range are initialized as Use the linearization formula to iteratively solve the longitude and latitude of the two calculation points Subsequently, the coordinates of each of the two calculation points in the coordinate system of the TIN model are calculated according to the longitude and latitude of the two calculation points, thereby calculating the vector of the photographic light of the i-th pixel.

[0073] For example, the coordinates of the two calculation points can be calculated respectively according to the following formula 1.

[0074]

[0075] Among them, v s 、v L are the residuals caused by the linearization of the model; L and S are the row and column image coordinates of the calculation point corresponding to the original image; λ are the geographical coordinates corresponding to the calculation point - longitude and latitude coordinates; Δλ are the correction values ​​of the geographic coordinates corresponding to the calculation points, r s 、r L is the model deviation of the initial value of the geographic coordinates corresponding to the calculation point.

[0076] It should be noted that the coordinates corresponding to the i-th pixel in the original image can be calculated based on the coordinates of the i-th pixel in the occlusion mask image and the DHM data. For example, the elevation of the i-th pixel in the DHM can be interpolated through the DMH data based on the coordinates of the i-th pixel in the occlusion mask image to obtain the ground coordinates of the i-th pixel in the coordinate system of the DHM; then, the ground coordinates corresponding to the i-th pixel are projected, and the projected coordinates are substituted into the RPC model for calculation, thereby obtaining the coordinates corresponding to the i-th pixel in the original image.

[0077] For example, if Figure 2 As shown, the intersection point A of the photographic light of the i-th pixel with the maximum elevation surface in the ground coverage range and the intersection point B with the lowest elevation surface are taken as calculation points. According to formula 1, the longitude and latitude coordinates of the intersection points A and B are calculated, and the longitude and latitude of the intersection points A and B are converted into a plane rectangular coordinate system. Subsequently, with point A as the starting point, a ray X passing through point B is constructed. According to the plane rectangular coordinate system coordinates of points A and B, the equation of ray AB can be calculated, and the equation of ray AB is used as the photographic light vector of the i-th pixel.

[0078] Step S107, using a ray tracing algorithm, according to the photographic ray vector of the i-th pixel and the TIN model, obtain the obscured pixel in the occlusion mask image; and set the value of the obscured pixel to the second value.

[0079] In the TIN model, for the photographic light of the ith pixel, starting from the photographic position, ray tracing is performed along the direction of the photographic light of the ith pixel. The intersection judgment algorithm of the algorithm is used to judge whether the light intersects with the triangles in the TIN model. For each intersecting triangle, the intersection point with the photographic light is calculated, and it is judged whether the intersection point is blocked by other intersection points; if the intersection point is blocked by other intersection points, that is, the pixel corresponding to the intersection point is blocked, the value of the blocked pixel is set to the second value.

[0080] For example, if Figure 2 As shown in the figure, the photographic light vector of the i-th pixel (i.e., light AB) is traced to obtain the intersection of light AB with the three triangles of the TIN model. The three intersection points are intersection C, intersection D, and intersection E. According to the distances of the three intersection points from the ray starting point A, intersection C is the closest intersection to the ray starting point A; the distances d between intersection D and intersection E and the ray starting point A are AD d AE Less than the distance d between the intersection point C and the starting point A of the ray AC , then the intersection points D and E are blocked, and the pixels corresponding to the intersection points D and E in the blocking mask image are considered to be blocked; 1 is used as the second value, and the values ​​of the pixels corresponding to the intersection points D and E in the blocking mask image are set to 1.

[0081] To improve the detection efficiency, in some possible implementations, Figure 3 As shown, according to steps S301-S307, the occluded pixels in the occlusion mask image are obtained.

[0082] Step S301, dividing the TIN model into a plurality of TIN blocks according to the photographic light vectors of the corner pixels and the DHM data, wherein the corner pixels are pixels in the occlusion mask image corresponding to the vertices of the original image.

[0083] Further, such as Figure 4 As shown, step S301 may include steps S401 to S405.

[0084] Step S401, calculating the maximum azimuth angle difference and the average azimuth angle of the photographic light of the original image vertex according to the photographic light vector of the corner pixel and the DHM data.

[0085] First, according to the method of step S107, the photographic light vectors of the pixels corresponding to the four vertex positions of the original image in the occlusion mask image can be obtained; then the coordinate system of the photographic light vectors of the four pixels is transformed into the coordinate system of the DHM, and the azimuths of the photographic light of the four pixels relative to the X-axis of the DHM coordinate system are calculated respectively to obtain the maximum azimuth difference and the average azimuth of the photographic light of the original image.

[0086] For example, if Figure 5As shown, on the XOY plane in the DHM coordinate system, the four vertices of the original image are P1, P2, P3, and P4 respectively; the azimuths of the photographic light of vertices P1, P2, P3, and P4 respectively; the azimuth α1 of the photographic light of vertex P1 - light 1 is the maximum azimuth, and the azimuth α2 of the photographic light of vertex P3 - light 2 is the minimum azimuth; the maximum azimuth difference is α1-α2.

[0087] Step S403, calculating the block width of the TIN model according to the maximum azimuth difference and the maximum ray length of the ground coverage area.

[0088] Calculate the distance between each of the four vertices, and use the maximum vertex distance as the maximum ray length of the ground coverage. Then, according to the maximum ray length of the ground coverage and the maximum azimuth difference of the vertex photography ray, determine the diameter d of the possible landing area of ​​the ground point as the block width of the TIN model; for example, because the maximum azimuth difference of the vertex photography ray is small, the product of the maximum ray length of the ground coverage and the maximum azimuth difference can be used as the diameter d.

[0089] For example, if Figure 5 As shown, if light 1 and light 2 are translated to point P4, the maximum azimuth angle difference of the vertex photography light is Figure 5 Δα; According to the maximum ray length ρ of the ground coverage area and the maximum azimuth difference Δα, the block width d of the TIN model is calculated.

[0090] Step S405, dividing the TIN model into blocks according to the block width of the TIN model and the photographic light direction corresponding to the average azimuth angle.

[0091] First, the bounding rectangle of the ground coverage area is constructed according to the direction of the photographic light corresponding to the average azimuth angle. Figure 6 As shown, in the plane coordinate system XOY of the DHM model, the direction of the photographic light corresponding to the average azimuth angle α0, and the direction of the vertical line of the photographic light; establish Figure 6 The circumscribed rectangle A1A2A3A4 is shown.

[0092] Secondly, according to the block width of the TIN model, the circumscribed rectangle blocks are processed along the vertical direction of the photographic light corresponding to the average azimuth angle to obtain the circumscribed rectangle blocks. Figure 6 As shown, along the perpendicular direction of the photographic light corresponding to the average azimuth angle, that is, the direction of the straight line where A1A2 is located, the circumscribed rectangle A1A2A3A4 is divided into blocks with a width d; wherein, a small amount of overlap may be maintained between adjacent blocks of the circumscribed rectangle.

[0093] Finally, the TIN model is divided into TIN blocks according to the overlapping area between the bounding rectangle block and the ground coverage. Figure 6 As shown, in the first two circumscribed rectangular blocks, there are areas P1a1a2 and b1b2b3b4 overlapping with the ground coverage area, respectively. The ground coverage area is divided into blocks according to the overlapping areas to obtain TIN blocks P1a1a2 and b1b2b3b4.

[0094] Step S303 , obtaining the TIN block corresponding to the i-th pixel from the TIN model according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel.

[0095] In a more specific implementation, the TIN block corresponding to the i-th pixel may be selected from the TIN model according to the coordinate of the i-th pixel in the occlusion mask image and the principle of being closest to the center line.

[0096] According to the coordinates of the ith pixel in the occlusion mask image, the ground coordinates of the ith pixel in the coordinate system where the DHM is located are obtained; according to the ground coordinates of the ith pixel, the nearest block is selected according to the distance center line principle.

[0097] For example, according to the coordinates (R, C) of the ith pixel in the occlusion mask image, the ground coordinates (X, Y, Z) of the ith pixel in the coordinate system where the DHM is located can be obtained according to the above steps. Subsequently, according to the distance between the ground coordinates (X, Y, Z) of the ith pixel and the center lines of each TIN block, the TIN block corresponding to the ith pixel is selected according to the principle of the closest distance to the center line.

[0098] Step 305 , using a ray tracing algorithm, according to the photographic ray vector of the i-th pixel and the TIN block corresponding to the i-th pixel, obtain the obscured pixel in the occlusion mask image.

[0099] When using the ray tracing algorithm to perform ray tracing, only the triangles contained in the TIN block corresponding to the i-th pixel obtained in step S303 can be used, and an intersection judgment algorithm can be used to judge whether the light intersects with the triangles in the TIN block to judge whether the photographic light for the i-th pixel is blocked. By using the photographic light azimuth to constrain the number of triangles in the TIN model, the speed of occlusion detection is greatly improved while ensuring the accuracy of occlusion detection.

[0100] In a more specific implementation, a triangle within the projection range of the photographic light of the ith pixel can be intercepted from the TIN block corresponding to the ith pixel to obtain a triangle set; based on the triangle set and the photographic light vector of the ith pixel, a ray tracing algorithm is used to obtain the occluded pixel in the occlusion mask image.

[0101] For example, first, according to the ground coordinates of the ith pixel, the TIN block corresponding to the ith pixel is selected from the TIN model according to the principle of the nearest center line; then, from the TIN block corresponding to the ith pixel, the triangle within the range of the photographic light AB of the ith pixel is intercepted to form the following: Figure 2 The triangle set shown in the dark part; using the triangle set and the photographic light of the i-th pixel, it is determined whether the photographic light of the i-th pixel is blocked.

[0102] It should be noted that the triangulated network within the projection range of the photographic light of the ith pixel can also be intercepted according to the maximum and minimum elevation values ​​existing in the TIN block of the ith pixel to obtain a triangle set.

[0103] As shown in the above process, when all pixels of the occlusion mask image are detected, the occlusion mask image is output. The occlusion mask is a binary image, and the pixel value of the occluded pixel is the second value, that is, the pixel value of the occluded pixel in the above example is 1; the value of other pixels is the first value, that is, the pixel value of other pixels in the above example is 0. In this way, occlusion detection of the original image can be achieved. Figure 7 An occlusion mask image obtained by this method is shown.

[0104] Based on the same concept as the above method embodiment, Figure 8 As shown, an image occlusion detection device is also provided in the embodiment of this specification. The device includes: a first calculation unit 801, a model construction unit 803, an image construction unit 805, a light calculation unit 807, and an occlusion judgment unit 809.

[0105] The first calculation unit 801 is used to calculate the ground coverage of the original image according to the original image and its corresponding RPC data;

[0106] A model building unit 803 is used to build a TIN model according to the ground coverage and its corresponding DHM data;

[0107] An image construction unit 805 is used to construct an occlusion mask image according to the ground coverage range, wherein the value of each pixel in the occlusion mask image is a first value;

[0108] A second calculation unit 807 is used to calculate, for any i-th pixel in the occlusion mask image, a photographic light vector of the i-th pixel according to the DHM data and the coordinates of the i-th pixel corresponding to the original image;

[0109] The occlusion judgment unit 809 is used to obtain the occluded pixel in the occlusion mask image according to the photographic ray vector of the i-th pixel and the TIN model by using a ray tracing algorithm, and set the value of the occluded pixel to a second value.

[0110] The embodiments of this specification also provide a computing device, including a processor, wherein when the processor executes computer instructions / codes stored in a memory, the method described in any embodiment of this specification is implemented. The memory may be located in the computing device or outside the computing device.

[0111] The embodiments of the present specification also provide a computer-readable storage medium on which computer instructions / codes are stored. When the computer instructions / codes are executed in a computing device, the computing device executes the method described in any one of the embodiments of the present specification.

[0112] Those skilled in the art should be aware that in one or more of the above examples, the functions described in this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, the computer programs corresponding to these functions can be stored in a computer-readable medium or transmitted as one or more instructions / codes on a computer-readable medium, so that when the computer programs corresponding to these functions are executed by a computer, the method described in any embodiment of the present invention is implemented by the computer.

[0113] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0114] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image occlusion detection method, characterized in that: The method comprises: Calculate the ground coverage of the original image according to the original image and its corresponding rational polynomial coefficient RPC data; Constructing an irregular triangulated network (TIN) model according to the ground coverage and its corresponding digital height model (DHM) data; Constructing an occlusion mask image according to the ground coverage range, wherein the pixel value of each pixel in the occlusion mask image is a first value; Calculate the ground coordinates of a first intersection point of the photographic light of the i-th pixel with the maximum elevation surface in the ground coverage range and a second intersection point of the minimum elevation surface according to the maximum and minimum values ​​in the DHM data and the coordinates of the i-th pixel corresponding to the original image; Calculate the longitude and latitude of the first intersection point and the second intersection point according to the ground coordinates of the first intersection point and the second intersection point; and calculate the coordinates of the first intersection point and the second intersection point in the TIN model coordinate system according to the longitude and latitude of the first intersection point and the second intersection point; Calculating the vector of the photographic light of the i-th pixel according to the coordinates of the first intersection point and the second intersection point in the TIN model coordinate system; Dividing the TIN model into a plurality of TIN blocks according to the photographic light vectors of the corner pixels and the DHM data; the corner pixels are pixels in the occlusion mask image corresponding to the vertices of the original image; Obtaining a TIN block corresponding to the i-th pixel from the TIN model according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel; By using a ray tracing algorithm, according to the photographic ray vector of the i-th pixel and the TIN block corresponding to the i-th pixel, the obscured pixel in the occlusion mask image is obtained, and the pixel value of the obscured pixel is set to a second value.

2. The method according to claim 1, wherein: The step of dividing the TIN model into a plurality of TIN blocks according to the photographic light vector of the corner pixel and the DHM data includes: Calculating the maximum azimuth angle difference and the average azimuth angle in the photographic light of the original image according to the photographic light vector of the corner pixel and the DHM data; Calculating the block width of the TIN model according to the maximum azimuth difference and the maximum ray length of the ground coverage range; The TIN model is divided into a plurality of TIN blocks according to the block width of the TIN model and the photographic light direction corresponding to the average azimuth angle.

3. The method according to claim 2, wherein: The step of dividing the TIN model into a plurality of TIN blocks according to the block width of the TIN model and the photographic light direction corresponding to the average azimuth angle includes: Constructing a circumscribed rectangle of the ground coverage area according to the direction of the photographic light corresponding to the average azimuth angle; According to the block width of the TIN model, the circumscribed rectangular block is divided into a plurality of circumscribed rectangular blocks along the perpendicular direction of the photographic light corresponding to the average azimuth angle; The TIN model is divided into TIN blocks according to the overlapping area between the circumscribed rectangular block and the ground coverage area.

4. The method according to claim 1, wherein: Obtaining a TIN block corresponding to the i-th pixel from the TIN model according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel, including: According to the coordinates of the i-th pixel in the occlusion mask image and the DHM data, and in accordance with the principle of being closest to the center line, a TIN block corresponding to the i-th pixel is selected from the TIN model.

5. The method according to claim 1, wherein the step of obtaining the occluded pixel in the occlusion mask image by using a ray tracing algorithm according to the photographic ray vector of the i-th pixel and the TIN block corresponding to the i-th pixel comprises: From the TIN block corresponding to the i-th pixel, intercept the triangle within the projection range of the photographic light of the i-th pixel to obtain a triangle set; According to the triangle set and the photographic ray vector of the i-th pixel, a ray tracing algorithm is used to obtain the occluded pixel in the occlusion mask image.

6. An image occlusion detection device, characterized in that: The device comprises: A first calculation unit, configured to calculate the ground coverage of the original image according to the original image and its corresponding rational polynomial coefficient RPC data; A model building unit, used for building a TIN model according to the ground coverage and its corresponding DHM data; An image construction unit, configured to construct an occlusion mask image according to the ground coverage, wherein the pixel value of each pixel in the occlusion mask image is a first value; A second calculation unit is used to calculate the ground coordinates of a first intersection point of the photographic light of the i-th pixel with the maximum elevation surface and a second intersection point of the minimum elevation surface in the ground coverage range according to the maximum value and the minimum value in the DHM data and the coordinates of the i-th pixel corresponding to the original image; calculate the longitude and latitude of the first intersection point and the second intersection point according to the ground coordinates of the first intersection point and the second intersection point; and calculate the coordinates of the first intersection point and the second intersection point in the TIN model coordinate system according to the longitude and latitude of the first intersection point and the second intersection point; calculate the vector of the photographic light of the i-th pixel according to the coordinates of the first intersection point and the second intersection point in the TIN model coordinate system; The occlusion judgment unit is used to divide the TIN model into a plurality of TIN blocks according to the photographic light vector of the corner pixel and the DHM data; the corner pixel is a pixel in the occlusion mask image corresponding to the vertex position of the original image; according to the coordinates of the i-th pixel in the occlusion mask image and the photographic light vector of the i-th pixel, the TIN block corresponding to the i-th pixel is obtained from the TIN model; using a ray tracing algorithm, according to the photographic light vector of the i-th pixel and the TIN block corresponding to the i-th pixel, the occluded pixel in the occlusion mask image is obtained, and the pixel value of the occluded pixel is set to a second value.

7. A computing device, comprising a processor, wherein when the processor executes computer instructions / codes stored in a memory, the method of any one of claims 1-5 is implemented.

8. A computer-readable storage medium having computer instructions / codes stored thereon, wherein when the computer instructions / codes are executed in a computing device, the computing device executes the method according to any one of claims 1 to 5.

Images