Image stitching method, device and related products

Abstract

The embodiment of the present application discloses an image stitching method, device and related products. The method includes: determining the latitude and longitude parameters of pixels in multiple images to be processed, and the multiple images to be processed are related to the target area; based on the mapping relationship between the longitude and latitude and the grid coordinate system at the target stitching level, determining the grid coordinates corresponding to the pixels in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the pixels; the stitching level is positively correlated with the clarity of the stitched image; according to the grid coordinates corresponding to the pixels in the multiple images to be processed, determining overlapping pixels from the multiple images to be processed; according to the grid coordinates corresponding to the pixels in the multiple images to be processed and the overlapping pixels, stitching the pixels in the multiple images to be processed into the grid coordinate system to obtain the target image of the target area at the stitching level. The scenarios in which the solution can be applied include but are not limited to map navigation, autonomous driving, vehicle-road collaboration, vehicle networking and other scenarios.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image stitching method, device and related products. Background Art

[0002] Currently, map accuracy directly impacts the effectiveness of the technologies used in various map applications, including navigation, vehicle-road collaboration, autonomous driving, and smart agriculture. Traditionally generated maps, due to their low precision, are no longer sufficient for practical applications. To improve map accuracy, detection equipment such as LiDAR can be used to detect and image the areas where high-precision maps are needed.

[0003] Because the overall scope of an area may exceed the range that can be detected by a single detection device, it is often necessary to deploy multiple detection devices and obtain multiple images of the area accordingly. When it is necessary to present the collection effect of regional information in mapping operations, one way is to directly display multiple images separately. Since there are often overlapping parts in different images of the area, after the display, it is necessary to manually distinguish the images with lower overlap, so that the naked eye can more intuitively obtain the required information from the images. However, this display method is not visually friendly, tests the visual resolution ability of the operators, and wastes time in obtaining effective information.

[0004] Stitching multiple images together can improve the overall presentation of scene information and create a more visually appealing experience. However, this approach relies on identifying similar feature points in different point cloud images and stitching them together based on their similarity. This image stitching method often requires multiple iterations of machine learning, resulting in complex algorithms, high implementation difficulty, unstable stitching results, and a long time-consuming process. Summary of the Invention

[0005] The embodiments of the present application provide an image stitching method, device, and related products, which implement image stitching in a simpler manner, improve the stability of the stitching effect, and save stitching time.

[0006] In view of this, the first aspect of the present application provides an image stitching method, the method comprising:

[0007] Determining latitude and longitude parameters of at least one first pixel in a first image and latitude and longitude parameters of at least one second pixel in a second image, the first image and the second image being associated with a target area;

[0008] Based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, determining a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and determining a grid coordinate corresponding to the at least one second pixel in the grid coordinate system according to the longitude and latitude parameters of the at least one second pixel; the target stitching level is positively correlated with the clarity of the stitched image;

[0009] determining overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel;

[0010] According to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, the at least one first pixel and the at least one second pixel are spliced ​​into the grid coordinate system to obtain a target image of the target area at the target splicing level.

[0011] A second aspect of the present application provides an image stitching device, comprising:

[0012] a pixel longitude and latitude determination unit, configured to determine longitude and latitude parameters of at least one first pixel in a first image and longitude and latitude parameters of at least one second pixel in a second image, wherein the first image and the second image are associated with a target area;

[0013] a pixel-to-grid mapping unit, configured to determine, based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and to determine, based on the longitude and latitude parameters of the at least one second pixel, a grid coordinate corresponding to the at least one second pixel in the grid coordinate system; the target stitching level is positively correlated with the clarity of the stitched image;

[0014] an overlapping pixel determining unit, configured to determine overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel;

[0015] An image stitching unit is used to stitch the at least one first pixel and the at least one second pixel into the grid coordinate system according to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, so as to obtain a target image of the target area at the target stitching level.

[0016] A third aspect of the present application provides a computer device, comprising a processor and a memory:

[0017] The memory is used to store program codes and transmit the program codes to the processor;

[0018] The processor is configured to execute the steps of the image stitching method of the first aspect according to the instructions in the program code.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the image stitching method of the first aspect.

[0020] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0021] The image stitching method provided in the embodiment of the present application extracts pixels from the image to be processed, maps them to a grid coordinate system at a target stitching level based on their latitude and longitude parameters, obtains the grid coordinates corresponding to the pixels in the grid coordinate system, determines the overlapping pixels in multiple images to be processed based on the grid coordinates, and then stitches the pixels in the multiple images to be processed into the grid coordinate system based on the grid coordinates and overlapping pixels corresponding to the pixels in the multiple images to be processed, thereby obtaining a target image of the target area at the target stitching level. The target stitching level is positively correlated with the clarity of the stitched image, that is, the higher the target stitching level, the higher the image clarity. Therefore, the target stitching level can be customized based on the requirements for stitching clarity, thereby meeting various requirements for target image clarity. This solution is simple to implement and the stitching is precise, so the stitching effect is more stable. In addition, since there is no need to use complex algorithm training models and iterations, the time-consuming stitching of multiple images containing overlapping areas is saved, and a high-complete, read-friendly stitched image can be obtained more quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a set of images provided for direct presentation of this application;

[0023] Figure 2 A flowchart of an image stitching method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the distribution of tiles in a secondary tile map coordinate system provided by this application;

[0025] Figure 4 A flowchart of another image stitching method provided in an embodiment of the present application;

[0026] Figure 5 A partial schematic diagram of a low-level tile image provided in an embodiment of the present application;

[0027] Figure 6 A partial schematic diagram of a high-level tile image provided in an embodiment of the present application;

[0028] Figure 7 The embodiment of this application provides Figure 1 The effect diagram of the stitched images to be processed is shown;

[0029] Figure 8 This is an effect image of the tile image provided in the embodiment of the present application superimposed on the base image;

[0030] Figure 9A A schematic diagram of each stage of executing the image stitching method provided in an embodiment of the present application;

[0031] Figure 9B A schematic structural diagram of an image stitching device provided in an embodiment of the present application;

[0032] Figure 10 A schematic diagram of the structure of a server provided in an embodiment of the present invention;

[0033] Figure 11 A schematic structural diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] When detecting regional information, multiple images are often obtained because the regional range may be large. Due to the overlap of the detection range of the acquisition equipment, there may be overlapping areas in different images. To present the detected regional information, multiple images can be displayed directly or after image stitching. Figure 1 For example, Figure 1 A schematic diagram of a set of images provided for direct display in this application. Figure 1 As you can see, there are overlapping areas in the multiple images displayed here. This requires manual identification of the overlapping areas when viewing the image set, which slows down the process of obtaining effective information. This method is inconvenient and provides a poor visual experience. Stitching the images together can improve the viewing experience.

[0036] Current image stitching technologies match feature points in different images and stitch them together based on these matching points. This often requires machine learning to generate relevant models. Due to the complexity of the algorithms, both training and deployment are time-consuming. Furthermore, due to the complexity and implementation challenges of these algorithms, ensuring high-quality stitching is difficult.

[0037] In the embodiment of the present application, an image stitching method, device and related products are proposed. In view of the problem that the current image stitching is complex and time-consuming, a simple and fast solution is proposed. In the image stitching method provided in the embodiment of the present application, for a plurality of images to be processed, pixels in the image are extracted, and they are mapped to a grid coordinate system at a specific level according to their latitude and longitude parameters, overlapping pixels are found in the grid coordinate system, and the position association of the grid corresponding to the original pixel in the grid coordinate system and the identified overlapping pixels are combined to realize the stitching of pixels in the multiple overlapping images to be processed at the stitching level. The stitching level can be customized, thus meeting various requirements for the clarity of the stitched image (target image). It is simple to implement, the stitching effect is stable, and the time consumed in image stitching is saved.

[0038] For easier understanding, see Figure 2 , Figure 2 Flowchart of the image stitching method provided in the embodiment of this application. Figure 2 As shown, the image stitching method includes:

[0039] S201: Determine latitude and longitude parameters of pixels in a plurality of images to be processed.

[0040] In the embodiment of the present application, the area where high-precision mapping is required can be defined as a target area. As an example, the target area can be a residential area or a commercial street, etc., which is not limited here. In the embodiment of the present application, multiple images to be processed related to the target area are obtained in advance.

[0041] In order to detect the target area with high precision, the target area can be detected by high-precision detection equipment such as laser radar. In one possible implementation, a fixed detection device may not be able to fully detect the target area, so detection devices are deployed at multiple different points. The detection data of the target area by a detection device is used to generate one or more images to be processed. In another possible implementation, the detection device is movable, for example, the detection device is installed on a mobile vehicle, and the target area is detected according to a preset period during the movement of the detection device, or the target area is detected when the detection device reaches a preset point during the movement. As an example, the data obtained from the detection of each preset point is used to generate an image to be processed.

[0042] Here, the image to be processed is a two-dimensional image. In one possible implementation, the detection data of the target area detected by the detection device directly generates a two-dimensional image to be processed. In another possible implementation, the detection data of the target area detected by the detection device is three-dimensional data, such as a three-dimensional point cloud detected by a lidar. This three-dimensional data needs to be processed before the two-dimensional image to be processed is obtained.

[0043] In the image to be processed, the latitude and longitude parameters of each pixel are available. The following describes an optional implementation method for determining the latitude and longitude parameters of pixels in the target image to be processed. Here, the target image to be processed can be any one of the multiple images to be processed.

[0044] For the target image to be processed, the longitude and latitude parameters of the center point of the target image to be processed are obtained, as well as the offset of the pixels in the target image to be processed relative to the center point in the Mercator coordinate system. The longitude and latitude parameters of the center point of the target image to be processed and the offset of the pixels in the target image to be processed relative to the center point in the Mercator coordinate system can both be directly obtained after the detection device collects the data used to generate the target image to be processed. Based on the conversion relationship between longitude and latitude coordinates and Mercator coordinates, the longitude and latitude parameters of the center point can be converted into the Mercator coordinate system to obtain the coordinates of the center point in the Mercator coordinate system. At this point, the center point and the offset of the pixels relative to the center point are both located in the same coordinate system, namely the Mercator coordinate system. Next, the coordinates of the pixels in the Mercator coordinate system can be obtained based on the coordinates of the center point in the Mercator coordinate system and the offset of the pixels relative to the center point in the Mercator coordinate system. To obtain the longitude and latitude parameters of the pixel, the pixel coordinates in the Mercator coordinate system can be converted to longitude and latitude coordinates based on the conversion relationship between Mercator coordinates and longitude and latitude coordinates, ultimately obtaining the longitude and latitude parameters of the pixel in the target image to be processed. The following formula describes this process.

[0045] base_x=lng c *20037508.342789 / 180 formula (1)

[0046] base_y'=(log2(tan(90+lat c *pi / 360))) / (pi / 180) Formula (2)

[0047] base_y=base_y'*20037508.342789 / 180 Formula (3)

[0048] In the above formulas (1)-(3), lng c and lat cRespectively represent the longitude and latitude of the center point of the target processing image; the coordinates of the center point in the Mercator coordinate system are (base_x, base_y), and base_y' is the intermediate variable for converting the center point latitude to the Mercator coordinate system. The above formulas (1)-(3) realize the conversion of the longitude and latitude parameters of the center point to the coordinates of the center point in the Mercator coordinate system. That is, the above formulas (1)-(3) jointly describe the conversion relationship between longitude and latitude coordinates and Mercator coordinates.

[0049] m_x=base_x+dif_x Formula (4)

[0050] m_y=base_y-dif_y Formula (5)

[0051] In formula (4) and formula (5), the offset of the pixel relative to the center point in the Mercator coordinate system is expressed as (dif_x, dif_y), and the coordinates of the pixel in the Mercator coordinate system are expressed as (m_x, m_y).

[0052] lat=m_x*180 / 20037508.342789 Formula (6)

[0053] lng'=m_y*180 / 20037508.342789 Formula (7)

[0054] lng=(2*atan(exp(lng'*pi / 180))-pi / 2)*180 / pi Formula (8)

[0055] In the above formulas (6)-(8), lng and lat represent the longitude and latitude parameters of the pixel, respectively; lng' is the intermediate variable for converting the pixel's Mercator coordinates to the pixel's longitude. The above formulas (6)-(8) achieve the conversion from the pixel's Mercator coordinates to the pixel's longitude and latitude. That is, the above formulas (6)-(8) jointly describe the conversion relationship between Mercator coordinates and latitude and longitude coordinates.

[0056] In practical applications, the implementation method for determining the latitude and longitude parameters of pixels in the image to be processed is not limited to the above method. For example, if high-precision map operations need to be performed based on 84 longitude and latitude coordinates, or if high-precision map operations need to be performed based on 02 longitude and latitude coordinates, the conversion relationship between coordinate systems can also be used to obtain the latitude and longitude parameters of the pixels in the required coordinate system in this step.

[0057] In order to facilitate understanding of the implementation of image stitching, the following text uses the first image and the second image in multiple images to be processed as an example to implement image stitching. For example, the first image and the second image can be two images whose corresponding data are collected at a short time interval or whose collection positions are relatively close. For the sake of distinction and clear description, the pixels in the first image are defined as first pixels, and the pixels in the second image are defined as second pixels. When image stitching is performed based on the first image and the second image, the step described in S201 can specifically be to determine the longitude and latitude parameters of at least one first pixel in the first image and the longitude and latitude parameters of at least one second pixel in the second image.

[0058] S202: Based on the mapping relationship between the longitude and latitude and the grid coordinate system at the target stitching level, determine the grid coordinates corresponding to the pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the pixel.

[0059] In the embodiments of the present application, the stitching level is positively correlated with the clarity of the stitched image. The higher the stitching level, the clearer the stitched image of the same area. For example, when the stitching level is 20, the clarity of the stitched image is higher than when the stitching level is 17.

[0060] In addition, the stitching level is also associated with the grid coordinate system. According to the stitching level, the grid coordinate system under the stitching level can be obtained. The grid coordinate system under the stitching level mentioned in the embodiment of the present application can cover the aforementioned target area. The stitching level can determine the number and size of grids in the grid coordinate system under the stitching level. In the longitude direction of the grid coordinate system, the number of grids is positively correlated with the stitching level. Similarly, in the latitude direction of the grid coordinate system, the number of grids is also positively correlated with the stitching level. In the grid coordinate system, the size of the grid in the longitude direction and the latitude direction is the same.

[0061] The grid coordinate system under the stitching level mentioned in the embodiment of the present application can cover the aforementioned target area. The grid coordinate system in the embodiment of the present application has a definite coverage range. For example, the maximum longitude, minimum longitude, maximum latitude and minimum latitude of the range covered by the grid coordinate system under the stitching level are determined. In addition, due to the grid division in the grid coordinate system, the range corresponding to each grid is also determined, for example, the maximum longitude, minimum longitude, maximum latitude and minimum latitude covered by each grid are determined. This solution selects a target stitching level from a variety of optional stitching levels based on actual stitching requirements, and then performs image stitching based on the target stitching level.

[0062] As an example, if the longitude and latitude of a pixel fall within the coverage of a grid with a sequence of 2 in the longitude direction and 3 in the latitude direction in the grid coordinate system at the target stitching level, the grid coordinates corresponding to this pixel in the grid coordinate system can be determined to be (2, 3). Of course, the grid coordinates can also be expressed in the latitude direction first and then in the longitude direction. The specific grid coordinate representation method can be set according to actual needs and is not limited here.

[0063] The above merely illustrates a direct representation of the mapping relationship between longitude and latitude and the grid coordinate system at the target splicing level. The mapping relationship between longitude and latitude and the grid coordinate system at the target splicing level can also be represented in other ways in other implementations. For example, the mapping relationship between the number of grid cells in the grid coordinate system and 1 degree in the longitude direction, and the mapping relationship between the number of grid cells in the grid coordinate system and 1 arc in the latitude direction. Therefore, in practical applications, there is no limitation on the representation method of the mapping relationship between longitude and latitude and the grid coordinate system at the target splicing level.

[0064] When image stitching is performed based on the first image and the second image, the steps described in S202 may specifically include: determining the grid coordinates corresponding to the at least one pixel in the grid coordinate system at the target stitching level based on the latitude and longitude parameters of the at least one first pixel, and determining the grid coordinates corresponding to the at least one second pixel in the grid coordinate system based on the latitude and longitude parameters of the at least one second pixel. In other words, for the latitude and longitude parameters of the first pixel and the second pixel obtained in S201, the grid coordinates corresponding to the pixels in the grid coordinate system at the target level have been obtained by executing S202.

[0065] S203: Determine overlapping pixels from the multiple images to be processed according to grid coordinates corresponding to pixels in the multiple images to be processed.

[0066] Since there may be overlapping areas between different images in the multiple images to be processed, there may be overlapping pixels between different images. In order to achieve the splicing of multiple images to be processed, it is necessary to determine the overlapping pixels. S203 is an important step before achieving image splicing. Since S202 maps the pixels in the images to be processed from their latitude and longitude coordinate systems to the grid coordinate system, in S203, the overlapping pixels in the multiple images to be processed can be determined specifically based on the grid coordinates of the pixels in the previous coordinate system at the target splicing level obtained previously.

[0067] In one possible implementation, pixels in the multiple images to be processed that have the same grid coordinates in the grid coordinate system at the target stitching level can be determined as a group of overlapping pixels. In the present embodiment, the number of pixels in a group of overlapping pixels is not limited. In a specific implementation, after executing S203, multiple groups of overlapping pixels can be obtained.

[0068] When performing image stitching based on the first image and the second image, the step described in S203 may specifically include determining overlapping pixels in the at least one first pixel and the at least one second pixel based on the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel. The set of overlapping pixels thus determined includes a first pixel and a second pixel, i.e., the overlapping pixels are from the first image and the second image, respectively. The determination of overlapping pixels is also performed based on the grid coordinates of the first pixel and the second pixel in the grid coordinate system at the target level.

[0069] S204: splicing the pixels in the multiple images to be processed into a grid coordinate system according to the grid coordinates corresponding to the pixels in the multiple images to be processed and the overlapping pixels, to obtain a target image of the target area at a target splicing level.

[0070] Because the pixels in multiple images to be processed are mapped to the same target grid coordinate system at the same stitching level, these pixels can be stitched into the aforementioned grid coordinate system based on the grid coordinates corresponding to the pixels in the images. This specifically utilizes the positional association of the grid coordinates in the grid coordinate system. For example, in the grid coordinate system, the pixel with grid coordinates (2, 2) needs to be stitched between the pixels with grid coordinates (2, 1) and (2, 3).

[0071] During implementation, stitching pixels in an image to a grid coordinate system may specifically refer to stitching pixel features of the pixels to corresponding grid coordinates in the grid coordinate system. As an example, if the image to be processed is a grayscale image, the pixel features may include grayscale values. For non-overlapping pixels, their grayscale values ​​in the source image to be processed may be directly used as the target grayscale values, that is, the original pixel features of the non-overlapping pixels are used as the target pixel features of the non-overlapping pixels. For the overlapping pixels determined above, the target grayscale value needs to be determined based on the grayscale values ​​of multiple pixels contained in a group of overlapping pixels, that is, the target pixel features of a group of overlapping pixels are obtained based on the original pixel features of a group of overlapping pixels.

[0072] Still taking grayscale values ​​as an example of pixel features, various possible implementations of obtaining target pixel features of a group of overlapping pixels based on original pixel features of a group of overlapping pixels are introduced.

[0073] In a possible implementation, a weighted average of the grayscale values ​​of a group of overlapping pixels in the image to be processed is used as the target grayscale value of the group of overlapping pixels.

[0074] In another possible implementation, the maximum grayscale value among the grayscale values ​​of a group of overlapping pixels in the image to be processed is used as the target grayscale value for the group of overlapping pixels. For example, if the detection device is a mobile lidar, during movement, the same area (e.g., a road sign) may be obscured by the vehicle it is carrying. This may result in the sign appearing with a smaller grayscale value in the resulting image A and a larger grayscale value in the resulting image B. The smaller grayscale value in image A may be due to the vehicle obscuring the sign while the detection device was collecting the detection data used to form image A. By the time the detection data for image B is generated, the vehicle has already left and no longer obscures the sign. Therefore, using the maximum grayscale value among the grayscale values ​​of a group of overlapping pixels in the image to be processed as the target grayscale value for the group of overlapping pixels helps to present more realistic and reliable details of the target area in the stitched image.

[0075] Since the target pixel features of each pixel in multiple images to be processed are obtained, the target pixel features of the pixels in the multiple images to be processed can be subsequently stitched into the grid coordinate system according to the positions of the grid coordinates corresponding to the pixels in the grid coordinate system to obtain the target image of the target area at the target stitching level.

[0076] When image stitching is performed based on the first image and the second image, the steps described in S204 may specifically be: based on the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel, and the overlapping pixels, the at least one first pixel and the at least one second pixel are stitched into the grid coordinate system to obtain the target image of the target area at the target stitching level. Of course, the basic image for image stitching (i.e., the image to be processed) is not limited to the first image and the second image, but may also include other more images, and the number of basic images is not limited here. In addition, stitching is not limited to stitching all the pixels in each image to be processed. For example, some pixels may have no drawing function and have no reference value in map operations, so they can be omitted.

[0077] The above is an image stitching method provided by an embodiment of the present application. The method extracts pixels from the image to be processed, maps them to the grid coordinate system under the target stitching level according to their latitude and longitude parameters, obtains the grid coordinates corresponding to the pixels in the grid coordinate system, determines the overlapping pixels in multiple images to be processed based on the grid coordinates, and then stitches the pixels in the multiple images to be processed into the grid coordinate system according to the grid coordinates and overlapping pixels corresponding to the pixels in the multiple images to be processed, and obtains the stitched image of the target area under the target stitching level (i.e., the target image). The target stitching level is positively correlated with the clarity of the stitched image, that is, the higher the target stitching level, the higher the image clarity. Therefore, the target stitching level can be customized based on the demand for stitching clarity, thereby meeting various demands for target image clarity. The solution is simple to implement and the stitching is accurate, so the stitching effect is more stable. In addition, since there is no need to use complex algorithm training models and iterations, the time-consuming stitching of multiple images containing overlapping areas is saved, and a high-completeness, stitched image suitable for reading can be obtained more quickly.

[0078] As mentioned in the previous embodiment, the image to be processed can be a two-dimensional grayscale image converted from three-dimensional point cloud data. The following describes an exemplary implementation of this process.

[0079] For example, the three-dimensional point cloud data of the target area obtained by a certain laser radar at a certain point can be subjected to data extraction. Since the data density of three-dimensional point cloud data is usually large, data extraction can more evenly sparse the original data, thereby reducing the amount of three-dimensional point cloud data that needs to be processed and improving the conversion speed. Another optional operation is filtering. By filtering, the deviated noise data in the three-dimensional point cloud data can be filtered out, reducing the impact of the noise data on the final two-dimensional grayscale image. In order to achieve the conversion from three-dimensional to two-dimensional, it is also necessary to aggregate the three-dimensional point cloud data along the z-axis (i.e., the direction perpendicular to the ground). Finally, two-dimensional data of the three-dimensional point cloud data can be formed. In the embodiment of the present application, in the process of converting three-dimensional point cloud data into a two-dimensional grayscale image, the execution order of data extraction, filtering and aggregation operations is not limited. For example, it is also possible to first aggregate it into two-dimensional data and then perform data extraction and filtering.

[0080] The grid coordinate system is mentioned in the above embodiment. In a possible implementation, the grid coordinate system can specifically be a tile map coordinate system. The tile map is a grid-type mapping of the real geographical scope. As an example, the entire earth is taken as the geographical scope that requires grid-type mapping, and then a tiled grid-type two-dimensional tile map coordinate system can be established based on the entire earth. The tile map coordinate system is associated with the target splicing level, and the target splicing level here can be used as the tile division level of the tile map coordinate system. The tile map coordinate system under the target splicing level includes tiles segmented according to the above target splicing level. Assuming that the target splicing level is k (k is 0 or a positive integer), the entire tile map coordinate system after segmentation includes 2 in the longitude direction. k tiles, and includes 2 in the latitude direction k When k = 0, it means that the tile map coordinate system contains only one tile and no segmentation is performed. This shows that the higher the target splicing level, the more tiles there are, and the number of tiles in the tile map coordinate system is positively correlated with the target splicing level. Figure 3 This is a schematic diagram of the distribution of tiles in the secondary tile map coordinate system. The tile segmentation effects of other target splicing levels are not shown in detail. You can refer to Figure 3 . Combined Figure 3 It can be seen that in the tile map coordinate system, each tile has a unique corresponding identifier, such as (0, 0), (0, 1), (0, 2), etc. Figure 3 The value before the comma in the tile identifier represents the tile's ordinal number in the latitudinal direction of the tile map coordinate system, and the value after the comma represents the tile's ordinal number in the longitudinal direction of the tile map coordinate system. Of course, in other implementations, the ordinal number in the longitude direction can also be placed before the comma and the ordinal number in the latitude direction can be placed after the comma.

[0081] Tiles of different target stitching levels have the same number of grids in the longitude direction and the same number of grids in the latitude direction. For example, a level 0 tile has 512×512 grids, and each level 2 tile also has 512×512 grids. The longitude dimension of the grid is the same as the latitude dimension, that is, the shape of the grid is square. The number of grids arranged in the longitude direction and the number of grids arranged in the latitude direction within the tile can be the same or different. When they are the same (for example, 512×512), the tile is square, and when they are different, the tile is rectangular. The shape of the tile is related to the coverage of the entire tile map coordinate system and is not limited here.

[0082] The following introduces another image stitching method based on the tile map coordinate system introduced above. Figure 4 , which is a flow chart of another image stitching method provided by an embodiment of the present application. Figure 4 The methods shown include:

[0083] S401: Determine latitude and longitude parameters of pixels in a plurality of images to be processed.

[0084] The implementation of S401 in this embodiment is substantially the same as that of S201 in the aforementioned embodiment, and therefore reference may be made to the relevant description of S201 in the aforementioned embodiment, which will not be repeated here.

[0085] S402: Obtain the total number of grids in the longitude direction of the tile map coordinate system based on the number of grids in the longitude direction of the tiles at the target splicing level and the target splicing level; and obtain the total number of grids in the latitude direction of the tile map coordinate system based on the number of grids in the latitude direction of the tiles at the target splicing level and the target splicing level.

[0086] As mentioned earlier, the number of cells in a tile's longitude and latitude directions is independent of the tile level. For example, a tile containing 512×512 cells has 512 cells in the longitude direction and 512 cells in the latitude direction. The target tile level affects the division of tiles in the tile map coordinate system, and therefore, the total number of cells in the tile map coordinate system.

[0087] In order to subsequently determine the grid coordinates of the pixel in the tile map coordinate system, it is necessary to obtain the total number of grids in the longitude direction and the total number of grids in the latitude direction of the tile map coordinate system in this step. In specific implementation, the product of the number of grids in the longitude direction of the tile and the number of tiles in the longitude direction of the tile map coordinate system at the target splicing level can be used as the total number of grids in the longitude direction of the tile map coordinate system. Similarly, the product of the total number of grids in the latitude direction of the tile and the number of tiles in the latitude direction of the tile map coordinate system at the target splicing level is used as the total number of grids in the latitude direction of the tile map coordinate system. As an example, assuming that the tile contains 512×512 grids and the target splicing level is k, the total number of grids in the longitude direction and the total number of grids in the latitude direction of the tile map coordinate system are both 512×2 k .

[0088] S403: According to the total number of rasters in the longitude direction of the tile map coordinate system and the total number of rasters in the latitude direction of the tile map coordinate system, the mapping relationship between the longitude and latitude and the grid coordinate system at the target splicing level is obtained; the mapping relationship between the longitude and latitude and the grid coordinate system at the target splicing level includes: a first mapping relationship and a second mapping relationship; the first mapping relationship includes: the number of rasters corresponding to 1 degree in the tile map coordinate system at the target splicing level; the second mapping relationship includes: the number of rasters corresponding to 1 arc in the tile map coordinate system at the target splicing level.

[0089] In the embodiment of the present application, the first mapping relationship is used to construct the association of pixels in the longitude direction of the tile map coordinate system. The second mapping relationship is used to construct the association of pixels in the latitude direction of the tile map coordinate system. The reason why the first mapping relationship is used to describe the association in the longitude direction and the second mapping relationship is used to describe the association in the latitude direction is that for the entire earth, all longitude lines are basically the same length, while latitude lines have different lengths, and the total arc length of the latitude lines is basically equal.

[0090] Specifically, the first mapping relationship is obtained based on the total number of grids in the longitude direction of the tile map coordinate system. The second mapping relationship is obtained based on the total number of grids in the latitude direction of the tile map coordinate system. The expressions of the first mapping relationship and the second mapping relationship are as follows:

[0091] pixel_per_degree=total_pixel_lng / 360 formula (9)

[0092] pixel_per_radian = total_pixel_lat / (2*pi) Formula (10)

[0093] In formulas (9) and (10), total_pixel_lng and total_pixel_lat represent the total number of grid cells in the longitude and latitude directions of the tile map coordinate system at the target stitching level. pixel_per_degree represents the number of grid cells corresponding to 1 degree in the tile map coordinate system at the target stitching level, and pixel_per_radian represents the number of grid cells corresponding to 1 radian in the tile map coordinate system at the target stitching level.

[0094] S404: Obtain the longitude grid ordinal number of the pixel in the tile map coordinate system according to the longitude parameter of the pixel, the first mapping relationship and the total number of grids in the longitude direction of the tile map coordinate system; and obtain the latitude grid ordinal number of the pixel in the tile map coordinate system according to the latitude parameter of the pixel, the second mapping relationship and the total number of grids in the latitude direction of the tile map coordinate system.

[0095] In an embodiment of the present application, the grid coordinates of the pixel in the image to be processed in the tile map coordinate system include two parts, one part is the longitude grid number of the pixel in the tile map coordinate system, and the other part is the latitude grid number of the pixel in the tile map coordinate system.

[0096] The implementation process of S404 is introduced below with reference to the formula.

[0097] c1=total_pixel_lng / 2 formula (11)

[0098] c2=total_pixel_lat / 2 Formula (12)

[0099] f=sin(lat*pi / 180) Formula (13)

[0100] x=c1+lng*pixel_per_degree formula (14)

[0101] y=c2-0.5(log2((1+f) / (1-f)))*pixel_per_radian Formula (15)

[0102] In formulas (11)-(15), the longitude and latitude parameters of a pixel are lng and lat, respectively, and the longitude and latitude grid numbers of the pixel in the tile map coordinate system are x and y, respectively. c1 is the intermediate quantity mapped from the longitude parameter of the pixel to the tile map coordinate system, and c2 and f are two intermediate quantities mapped from the latitude parameter of the pixel to the tile map coordinate system.

[0103] That is, the grid coordinates of a pixel in the tile map coordinate system can be expressed as (x, y).

[0104] When performing image stitching based on the first image and the second image, S401 may specifically determine the latitude and longitude parameters of at least one first pixel in the first image and the latitude and longitude parameters of at least one second pixel in the second image. The step described in S404 may specifically be for a target pixel, where the target pixel is any one of the at least one first pixel and the at least one second pixel.

[0105] S405: Determine overlapping pixels from the multiple images to be processed according to grid coordinates corresponding to pixels in the multiple images to be processed.

[0106] Specifically, overlapping pixels can be determined based on the grid coordinates of the tile map coordinate system of the pixels at the same target splicing level. The overlapping pixels have the same x and y coordinates.

[0107] In addition, as another possible implementation method, tiles can be used as the unit of the stitched image during subsequent stitching, that is, pixels mapped to the same tile are stitched together to obtain a stitched tile image. To this end, in order to identify the tile corresponding to the pixel in batch operations and facilitate the execution of subsequent stitching operations, further operations can be performed based on the grid coordinates (x, y). The following is a detailed description, taking the target pixel as an example:

[0108] Obtaining a first quotient and a first remainder of the longitude direction grid number of the target pixel to the longitude direction grid number, and obtaining a second quotient and a second remainder of the latitude direction grid number of the target pixel to the latitude direction grid number;

[0109] The identifier of the tile that the target pixel matches in the tile map coordinate system is obtained from the first quotient and the second quotient; the intra-tile coordinates that the target pixel matches in the tile map coordinate system are obtained from the first remainder and the second remainder.

[0110] The tile that matches the pixel in the tile map coordinate system at the target mosaic level is identified by ([x / pixel_size1], [y / pixel_size2]). Here, pixel_size1 and pixel_size2 are the number of grid cells in the tile in the longitude and latitude directions. Assuming a tile contains 512×512 grid cells, pixel_size1 = pixel_size2 = 512. The [] in the above notation is a rounding symbol, which is used to obtain the quotient of the dividend and divisor within the symbol.

[0111] The coordinates of the pixel in the tile map coordinate system at the target stitching level are expressed as (pix_x, pix_y), where pix_x = x% pixel_size1 and pix_y = y% pixel_size2. pix_x and pix_y correspond to the longitude and latitude directions, respectively.

[0112] When there are many tiles, overlapping pixels can be determined based on the identifiers of the tiles that match the pixels in the multiple images to be processed and the pixel's intra-tile coordinates. For example, pixels with the same identifier and intra-tile coordinates in the matching tiles in the multiple images to be processed can be considered a group of overlapping pixels. When performing image stitching based on the first and second images, the first and second pixels with the same identifier and intra-tile coordinates in the matching tiles of at least one first pixel and at least one second pixel can be considered a group of overlapping pixels.

[0113] In addition, if there are many tile stitching levels, such as level 1 tiles, level 2 tiles, level 3 tiles, etc., in order to achieve accurate stitching of pixels in the image at the same target stitching level, the identification of the pixel-matched tiles and the intra-tile coordinates of the pixels can be further combined with the pixel stitching level to determine the overlapping pixels corresponding to the same target stitching level, the same tile and the same intra-tile coordinates.

[0114] S406: Based on the intra-tile coordinates of the pixels of the same tile at the target stitching level in the multiple images to be processed and the overlapping pixels matching the same tile, the pixels matching the same tile in the images to be processed are stitched into the same tile to obtain a target image with the target area in the same tile.

[0115] As mentioned above, tiles can be used as the unit of the stitched image. Therefore, in a specific implementation, based on the intra-tile coordinates of pixels matching the same tile at the target stitching level in multiple images to be processed and the overlapping pixels matching the same tile, the pixels matching the same tile in the images to be processed can be stitched together into the same tile, thereby obtaining a target image with the target region in the same tile. For example, a target image can be stitched together from multiple tiles with different identifiers corresponding to the target region.

[0116] When image stitching is performed based on the first image and the second image, S406 can specifically be: according to the intra-tile coordinates of the pixels in the at least one first pixel and the at least one second pixel that match the same tile at the target stitching level and the overlapping pixels that match the same tile, the pixels in the at least one first pixel and the at least one second pixel that match the same tile are stitched into the same tile to obtain the target image of the target area in the same tile.

[0117] Figure 5 and Figure 6 This is a partial schematic diagram of tile images with two different target splicing levels provided in the embodiment of the present application. Figure 5 The tile image target stitching level is low, Figure 6 The tile image target stitching level is higher. Combined Figure 5 and Figure 6 It can be seen that the higher the target stitching level, the clearer the details of the target area that can be displayed and the higher the imaging accuracy.

[0118] Assumptions Figure 1 The set of images shown are multiple images to be processed mentioned in the embodiment of this application. Two tile images are obtained by splicing these images, such as Figure 7 shown. Figure 7 The embodiment of this application provides Figure 1 The image below shows the effect of the stitching of the image to be processed. It can be seen that the tile image can display the pixel features in the image to be processed in a tiled grid manner.

[0119] Tile images can be stored independently. Of course, based on the positional association of tiles in the tile map coordinate system, they can be further stitched together to obtain a more complete image of the target area at the target stitching level, which can then be stored. The tile image stitching process can be performed before storage or when the tile image is applied, and this is not limited here.

[0120] Since the target stitching level can be customized, tile images of the target area at multiple target stitching levels can be obtained. Since tiles of each target stitching level can be displayed by tiling, the reference efficiency of the original detection data (such as the three-dimensional point cloud of the lidar) for high-precision map operations can be greatly improved. The stitching processing of different target stitching levels can be executed in parallel, and the mapping conversion of different pixels to grid coordinate systems can also be executed in parallel. There is no restriction on the execution order here.

[0121] In practical applications, the tile images obtained above can assist in high-precision mapping of the target area. For example, a base map of the target area is obtained in advance. Here, the base map can refer to a satellite image of the target area. The base map is often of low clarity and lacks rich details. The data source of the tile images is often high-precision detection equipment, such as lidar. Therefore, the tile map can enrich the details of the target area and improve display accuracy and clarity. In specific implementations, the tile images can be superimposed on the base map to obtain the superimposed image. Figure 8 This is the effect diagram of the tile image provided in the embodiment of the present application being superimposed on the base image. Figure 8 As shown in the figure, the base image is relatively blurry and has poor clarity. After superimposing the tile image, the image clarity is significantly improved and the details are relatively complete.

[0122] It should be noted that due to the limitations of the detection method and detection route of the detection equipment, the final tile image may contain some grids with missing grayscale values. These grids do not have valid or corresponding pixels in the multiple images to be processed. Therefore, grids with missing grayscale values ​​may appear as 100%. Figure 8 This is the effect shown. Figure 8 A part of the base map is superimposed with tiles, showing an overlay effect, such as the details on the road are displayed more clearly and richly; another part of the base map can only be superimposed with tiles with 100% transparency, showing the original base map effect.

[0123] Furthermore, when working with high-precision maps, the base map and overlay effects may need to be displayed at a specific display magnification (i.e., target display magnification) based on the viewing needs of the staff. In this case, tile images that match the target display magnification need to be overlaid on the base map at the target display magnification. This ensures a matching visual effect and meets the viewing needs of the staff.

[0124] The specific implementation is as follows:

[0125] The target images of the target area at multiple different target stitching levels are stored in the image library, and the mapping relationship between different display magnifications and target stitching levels can be pre-established. According to the mapping relationship between the display magnification and the target stitching level, the target stitching level corresponding to the target display magnification is determined; the target image of the target area at the target stitching level (for example, the tile image of the target area at the target stitching level) is loaded from the image library. The loaded target image is then superimposed with the base map of the target area at the target display magnification. Finally, the image overlay effect is displayed. By pre-storing the target images of the target area at multiple target stitching levels, the viewing needs of the staff during high-precision map operations can also be met in real time, thereby improving the visual experience.

[0126] Figure 9A Schematic diagram of each stage of the image stitching method provided in an embodiment of the present application. First, the image to be processed is obtained, and then the latitude and longitude parameters of the pixels in the image to be processed are obtained. Then, the pixels are converted into rasterized tiles to obtain the pixel's tile internal coordinates and tile identification, thereby achieving a mapping between the pixel longitude and latitude parameters and the tile grid coordinates. Then, overlapping pixels are identified based on the obtained tile internal coordinates and tile identification, and tile images of corresponding tiles are stitched together based on the tile internal coordinates and overlapping pixels of the pixels in multiple images to be processed. Finally, the tile image is stored in the library for use.

[0127] Based on the image stitching method described in the foregoing embodiment, this application further introduces an implementation method of an image stitching device.

[0128] See also Figure 9B , which is a structural diagram of an image splicing device 90 provided in an embodiment of the present application. Figure 9B The image stitching device 90 shown includes:

[0129] a pixel longitude and latitude determination unit 91, configured to determine longitude and latitude parameters of at least one first pixel in a first image and longitude and latitude parameters of at least one second pixel in a second image, wherein the first image and the second image are associated with a target area;

[0130] a pixel-to-grid mapping unit 92 configured to determine, based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, the grid coordinates corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and to determine, based on the longitude and latitude parameters of the at least one second pixel, the grid coordinates corresponding to the at least one second pixel in the grid coordinate system; the target stitching level is positively correlated with the clarity of the stitched image;

[0131] an overlapping pixel determining unit 93, configured to determine overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel;

[0132] The image stitching unit 94 is used to stitch the at least one first pixel and the at least one second pixel into the grid coordinate system according to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel, and the overlapping pixels, to obtain the target image of the target area at the target stitching level.

[0133] The image stitching device provided in the embodiment of the present application extracts pixels from the image to be processed, maps them to a grid coordinate system at a stitching level based on their latitude and longitude parameters, obtains the grid coordinates corresponding to the pixels in the grid coordinate system, determines the overlapping pixels in multiple images to be processed based on the grid coordinates, and then stitches the pixels in the multiple images to be processed into the grid coordinate system based on the grid coordinates and overlapping pixels corresponding to the pixels in the multiple images to be processed, thereby obtaining a target image of the target area at a target stitching level. The target stitching level is positively correlated with the clarity of the stitched target image, that is, the higher the target stitching level, the higher the image clarity. Therefore, the target stitching level can be customized based on the requirements for stitching clarity, thereby meeting various requirements for target image clarity. This solution is simple to implement and the stitching is precise, so the stitching effect is more stable. In addition, since there is no need to use complex algorithm training models and iterations, the time spent on stitching multiple images containing overlapping areas is saved, and a high-complete, read-friendly stitched image can be obtained more quickly.

[0134] In a possible implementation, the overlapping pixel determining unit 93 is configured to determine, among the at least one first pixel and the at least one second pixel, the first pixel and the second pixel that have the same corresponding grid coordinates as a group of overlapping pixels;

[0135] The image stitching unit 94 includes:

[0136] a target pixel feature acquisition subunit, configured to obtain target pixel features of the group of overlapping pixels based on original pixel features of the group of overlapping pixels; and use the original pixel features of non-overlapping pixels as target pixel features of the non-overlapping pixels;

[0137] The pixel stitching subunit is used to stitch the target pixel features of the at least one first pixel and the target pixel features of the at least one second pixel into the grid coordinate system according to the positions of the grid coordinates corresponding to the pixels in the grid coordinate system, so as to obtain the target image of the target area at the target stitching level.

[0138] In a possible implementation, the first image and the second image are two-dimensional grayscale images converted from three-dimensional point cloud data of the target area; the pixel features include: grayscale value;

[0139] The target pixel feature acquisition subunit is specifically configured to use the maximum grayscale value of a group of overlapping pixels as the target grayscale value of the group of overlapping pixels.

[0140] In a possible implementation, the grid coordinate system is specifically a tile map coordinate system; the tile map coordinate system includes tiles; the number of tiles is positively correlated with the splicing level;

[0141] The mapping relationship between longitude and latitude and the grid coordinate system at the target splicing level includes: a first mapping relationship and a second mapping relationship; the first mapping relationship includes: the number of grid cells corresponding to 1 degree in the tile map coordinate system at the target splicing level; the second mapping relationship includes: the number of grid cells corresponding to 1 arc in the tile map coordinate system at the target splicing level;

[0142] The pixel and grid mapping unit 92 includes:

[0143] The grid coordinate first component acquisition subunit is used to obtain the longitude grid number of the target pixel in the tile map coordinate system according to the longitude parameter of the target pixel, the first mapping relationship and the total number of grids in the longitude direction of the tile map coordinate system;

[0144] The grid coordinate second component acquisition subunit is used to obtain the latitude grid number of the target pixel in the tile map coordinate system based on the latitude parameter of the target pixel, the second mapping relationship and the total number of grids in the latitude direction of the tile map coordinate system; the target pixel is any one of the at least one first pixel and the at least one second pixel.

[0145] In a possible implementation, the image stitching device 91 further includes:

[0146] a grid total number calculation unit, configured to obtain the total number of grids in the longitude direction of the tile map coordinate system based on the number of grids in the longitude direction of the tiles at the target splicing level and the target splicing level; and to obtain the total number of grids in the latitude direction of the tile map coordinate system based on the number of grids in the latitude direction of the tiles at the target splicing level and the target splicing level;

[0147] The mapping relationship acquisition unit is used to obtain a first mapping relationship according to the total number of grids in the longitude direction of the tile map coordinate system; and obtain a second mapping relationship according to the total number of grids in the latitude direction of the tile map coordinate system.

[0148] In a possible implementation, the overlapping pixel determining unit 93 includes:

[0149] An operator unit, configured to obtain a first quotient and a first remainder of the longitude direction grid number of the target pixel and the longitude direction grid number, and obtain a second quotient and a second remainder of the latitude direction grid number of the target pixel and the latitude direction grid number;

[0150] a tile identification acquisition subunit, configured to obtain an identification of a tile that matches the target pixel in the tile map coordinate system from the first quotient and the second quotient;

[0151] The tile intra-coordinate acquisition subunit is used to obtain the tile intra-coordinates of the target pixel matched in the tile map coordinate system from the first remainder and the second remainder;

[0152] The overlapping pixel determination subunit is configured to treat the first pixel and the second pixel having the same identifier of the matched tile and the same coordinate within the tile as a group of overlapping pixels.

[0153] In a possible implementation, the image stitching unit 94 includes:

[0154] The tile image stitching subunit is used to stitch the pixels in the at least one first pixel and the at least one second pixel that match the same tile at the target stitching level into the same tile based on the intra-tile coordinates of the pixels in the at least one first pixel and the at least one second pixel that match the same tile at the target stitching level and the overlapping pixels that match the same tile, so as to obtain the target image of the target area in the same tile.

[0155] In a possible implementation, the image stitching device 90 further includes:

[0156] A storage unit, used for storing target images of a target area at multiple different target stitching levels into an image library;

[0157] A target splicing level determining unit is configured to determine a target splicing level corresponding to a target display magnification according to a mapping relationship between the display magnification and the target splicing level;

[0158] An image loading unit, used for loading a target image of a target area at a target stitching level corresponding to a target display magnification from an image library;

[0159] An image overlay unit, used to overlay the loaded target image with a base map of the target area at a target display magnification;

[0160] The display unit is used to display the image superposition effect.

[0161] In a possible implementation, the pixel longitude and latitude determining unit 91 includes:

[0162] a center point longitude and latitude obtaining subunit, configured to obtain, for the first image, the longitude and latitude parameters of the center point of the first image;

[0163] an offset obtaining subunit, configured to obtain an offset of the at least one first pixel in the first image relative to the center point in a Mercator coordinate system;

[0164] The first conversion subunit is used to convert the latitude and longitude parameters of the center point into the Mercator coordinate system to obtain the coordinates of the center point in the Mercator coordinate system;

[0165] a pixel coordinate acquisition subunit, configured to obtain the coordinates of the at least one first pixel in the Mercator coordinate system based on the coordinates of the center point in the Mercator coordinate system and an offset of the at least one first pixel relative to the center point in the Mercator coordinate system;

[0166] The pixel longitude and latitude obtaining subunit is configured to obtain longitude and latitude parameters of the at least one first pixel in the first image according to the coordinate transformation of the at least one first pixel in the Mercator coordinate system.

[0167] The process of determining the latitude and longitude parameters of at least one second pixel in the second image is similar to the above and will not be repeated here.

[0168] The embodiment of the present application also provides a computer device, and the computer device provided by the embodiment of the present application will be introduced below from the perspective of hardware instantiation.

[0169] Figure 10 : This is a schematic diagram of a server structure provided by an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 922 (for example, one or more processors) and memories 932, and one or more storage media 930 (for example, one or more mass storage devices) for storing application programs 942 or data 944. Among them, the memories 932 and the storage media 930 can be temporary storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 922 can be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.

[0170] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0171] The steps performed by the server in the above embodiment can be based on the Figure 10 The server structure shown.

[0172] The CPU 922 is configured to execute the following steps:

[0173] Determining latitude and longitude parameters of at least one first pixel in a first image and latitude and longitude parameters of at least one second pixel in a second image, the first image and the second image being associated with a target area;

[0174] Based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, determining a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and determining a grid coordinate corresponding to the at least one second pixel in the grid coordinate system according to the longitude and latitude parameters of the at least one second pixel; the target stitching level is positively correlated with the clarity of the stitched image;

[0175] determining overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel;

[0176] According to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, the at least one first pixel and the at least one second pixel are spliced ​​into the grid coordinate system to obtain a target image of the target area at the target splicing level.

[0177] With respect to the image stitching method described above, an embodiment of the present application further provides a terminal device for image stitching, so that the above-mentioned image stitching method can be implemented and applied in practice.

[0178] See also Figure 11 , Figure 11This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present application. The terminal can be any terminal device including a mobile phone, tablet computer, personal digital assistant (English full name: Personal Digital Assistant, English abbreviation: PDA), sales terminal (English full name: Point of Sales, English abbreviation: POS), car computer, etc. Taking the terminal as a mobile phone as an example:

[0179] Figure 11 The block diagram shows a partial structure of a mobile phone related to the terminal provided in the embodiment of the present application. Figure 11 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will appreciate that Figure 11 The mobile phone structure shown in the figure does not constitute a limitation to the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0180] The following combination Figure 11 A detailed introduction to the various components of a mobile phone:

[0181] RF circuitry 1010 can be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink information from the base station and transmits it to processor 1080 for processing. It also transmits uplink data to the base station. Typically, RF circuitry 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, and more. RF circuitry 1010 can also communicate with the network and other devices via wireless communications. The above-mentioned wireless communications may use any communication standard or protocol, including but not limited to Global System of Mobile Communications (Global System of Mobile communication, English abbreviation: GSM), General Packet Radio Service (English full name: General Packet Radio Service, GPRS), Code Division Multiple Access (English full name: Code Division Multiple Access, English abbreviation: CDMA), Wideband Code Division Multiple Access (English full name: Wideband Code Division Multiple Access, English abbreviation: WCDMA), Long Term Evolution (English full name: Long Term Evolution, English abbreviation: LTE), email, Short Messaging Service (English full name: Short Messaging Service, SMS), etc.

[0182] The memory 1020 can be used to store software programs and modules. The processor 1080 executes the various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 1020 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0183] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch panel 1031) and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction and detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 1080. It can also receive commands sent by the processor 1080 and execute them. In addition, the touch panel 1031 can be implemented using various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 may further include other input devices 1032. Specifically, the other input devices 1032 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick.

[0184] The display unit 1040 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (English full name: Liquid Crystal Display, English abbreviation: LCD), an organic light-emitting diode (English full name: Organic Light-Emitting Diode, English abbreviation: OLED), etc. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 11 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0185] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the mobile phone can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0186] Audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the phone. Audio circuit 1060 converts received audio data into electrical signals and transmits them to speaker 1061, which then converts them into sound signals for output. Microphone 1062, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 1060 and converted into audio data. The audio data is then processed by processor 1080 and transmitted to, for example, another phone via RF circuit 1010, or stored in memory 1020 for further processing.

[0187] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web and access streaming media through the WiFi module 1070. It provides users with wireless broadband Internet access. Figure 11 A WiFi module 1070 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0188] Processor 1080 is the control center of the mobile phone, connecting all parts of the mobile phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 1020 and accessing data stored in memory 1020, it performs various functions of the mobile phone and processes data, thereby providing overall monitoring of the mobile phone. Optionally, processor 1080 may include one or more processing units; preferably, processor 1080 may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1080.

[0189] The mobile phone also includes a power supply 1090 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1080 through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0190] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0191] In the embodiment of the present application, the processor 1080 included in the terminal further has the following functions:

[0192] Determining latitude and longitude parameters of at least one first pixel in a first image and latitude and longitude parameters of at least one second pixel in a second image, the first image and the second image being associated with a target area;

[0193] Based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, determining a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and determining a grid coordinate corresponding to the at least one second pixel in the grid coordinate system according to the longitude and latitude parameters of the at least one second pixel; the target stitching level is positively correlated with the clarity of the stitched image;

[0194] determining overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel;

[0195] According to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, the at least one first pixel and the at least one second pixel are spliced ​​into the grid coordinate system to obtain a target image of the target area at the target splicing level.

[0196] An embodiment of the present application further provides a computer-readable storage medium for storing program code, wherein the program code is used to execute any one of the implementations of the image stitching method described in the aforementioned embodiments.

[0197] The embodiments of the present application further provide a computer program product including instructions, which, when executed on a computer, enables the computer to execute any one of the image stitching methods described in the aforementioned embodiments.

[0198] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0199] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0201] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program code.

[0203] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image stitching method, characterized in that: include: Determining latitude and longitude parameters of at least one first pixel in a first image and latitude and longitude parameters of at least one second pixel in a second image, the first image and the second image being associated with a target area; Based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, determining a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and determining a grid coordinate corresponding to the at least one second pixel in the grid coordinate system according to the longitude and latitude parameters of the at least one second pixel; the target stitching level is positively correlated with the clarity of the stitched image; determining overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel; According to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, the at least one first pixel and the at least one second pixel are spliced ​​into the grid coordinate system to obtain a target image of the target area at the target splicing level.

2. The method according to claim 1, characterized in that The determining, based on the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel, overlapping pixels in the at least one first pixel and the at least one second pixel includes: determining, among the at least one first pixel and the at least one second pixel, the first pixel and the second pixel having the same corresponding grid coordinates as a group of overlapping pixels; The step of stitching the at least one first pixel and the at least one second pixel into the grid coordinate system according to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel, and the overlapping pixels, to obtain a target image of the target area at the target stitching level includes: Obtain target pixel features of the group of overlapping pixels according to the original pixel features of the group of overlapping pixels; the original pixel features of non-overlapping pixels are used as the target pixel features of the non-overlapping pixels; The target pixel features of the at least one first pixel and the target pixel features of the at least one second pixel are associated and spliced ​​into the grid coordinate system according to the positions of the grid coordinates corresponding to the pixels in the grid coordinate system to obtain a target image of the target area at the target splicing level.

3. The method according to claim 2, characterized in that The first image and the second image are two-dimensional grayscale images converted from three-dimensional point cloud data of the target area; The pixel features include: grayscale value; The obtaining target pixel features of the group of overlapping pixels according to the original pixel features of the group of overlapping pixels comprises: The maximum grayscale value of the group of overlapping pixels is used as the target grayscale value of the group of overlapping pixels.

4. The method according to claim 1, wherein The grid coordinate system is specifically a tile map coordinate system; the tile map coordinate system includes tiles; The number of tiles is positively correlated with the target stitching level; The mapping relationship between the longitude and latitude and the grid coordinate system at the target splicing level includes: a first mapping relationship and a second mapping relationship; the first mapping relationship includes: the number of grids corresponding to 1 degree in the tile map coordinate system at the target splicing level; the second mapping relationship includes: the number of grids corresponding to 1 radian in the tile map coordinate system at the target splicing level; The determining, based on the mapping relationship between the longitude and latitude and the grid coordinate system at the target stitching level, the grid coordinates corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and the determining, based on the longitude and latitude parameters of the at least one second pixel, the grid coordinates corresponding to the at least one second pixel in the grid coordinate system, include: According to the longitude parameter of the target pixel, the first mapping relationship and the total number of grids in the longitude direction of the tile map coordinate system, the longitude grid number of the target pixel in the tile map coordinate system is obtained; and according to the latitude parameter of the target pixel, the second mapping relationship and the total number of grids in the latitude direction of the tile map coordinate system, the latitude grid number of the target pixel in the tile map coordinate system is obtained; the target pixel is any one of the at least one first pixel and the at least one second pixel.

5. The method according to claim 4, characterized in that Also includes: Obtaining the total number of grids in the longitude direction of the tile map coordinate system according to the number of grids in the longitude direction of the tile at the target splicing level and the target splicing level; and obtaining the total number of grids in the latitudinal direction of the tile map coordinate system according to the number of grids in the latitudinal direction of the tiles at the target splicing level and the target splicing level; The first mapping relationship is obtained according to the total number of grids in the longitude direction of the tile map coordinate system; and the second mapping relationship is obtained according to the total number of grids in the latitude direction of the tile map coordinate system.

6. The method according to claim 4, characterized in that The determining, based on the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel, overlapping pixels in the at least one first pixel and the at least one second pixel includes: Obtaining a first quotient and a first remainder of the longitude direction grid number of the target pixel to the longitude direction grid number, and obtaining a second quotient and a second remainder of the latitude direction grid number of the target pixel to the latitude direction grid number; Obtaining an identifier of a tile that matches the target pixel in the tile map coordinate system from the first quotient and the second quotient; obtaining intra-tile coordinates that match the target pixel in the tile map coordinate system from the first remainder and the second remainder; The first pixel and the second pixel having the same identifier of the matched tile and the same coordinates within the tile in the at least one first pixel and the at least one second pixel are regarded as a group of overlapping pixels.

7. The method according to claim 6, characterized in that The step of stitching the at least one first pixel and the at least one second pixel into the grid coordinate system according to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel, and the overlapping pixels, to obtain a target image of the target area at the target stitching level includes: According to the intra-tile coordinates of the pixels in the at least one first pixel and the at least one second pixel that match the same tile at the target stitching level and the overlapping pixels that match the same tile, the pixels in the at least one first pixel and the at least one second pixel that match the same tile are stitched into the same tile to obtain a target image of the target area in the same tile.

8. The method according to claim 1, characterized in that Also includes: storing target images of the target area at multiple different target stitching levels into an image library; According to the mapping relationship between the display magnification and the target splicing level, the target splicing level corresponding to the target display magnification is determined; Loading a target image of the target area at a target stitching level corresponding to the target display magnification from the image library; Overlaying the loaded target image with the base map of the target area at the target display magnification; Displays image overlay effects.

9. The method according to any one of claims 1 to 8, characterized in that The determining of the latitude and longitude parameters of at least one first pixel in the first image includes: For the first image, obtaining the latitude and longitude parameters of the center point of the first image, and obtaining the offset of the at least one first pixel in the first image relative to the center point in a Mercator coordinate system; Converting the latitude and longitude parameters of the center point into the Mercator coordinate system to obtain the coordinates of the center point in the Mercator coordinate system; Obtaining coordinates of the at least one first pixel in the Mercator coordinate system according to the coordinates of the center point in the Mercator coordinate system and an offset of the at least one first pixel relative to the center point in the Mercator coordinate system; The longitude and latitude parameters of the at least one first pixel in the first image are obtained according to the coordinate transformation of the at least one first pixel in the Mercator coordinate system.

10. An image stitching device, characterized in that: include: a pixel longitude and latitude determination unit, configured to determine longitude and latitude parameters of at least one first pixel in a first image and longitude and latitude parameters of at least one second pixel in a second image, wherein the first image and the second image are associated with a target area; a pixel-to-grid mapping unit, configured to determine, based on a mapping relationship between longitude and latitude and a grid coordinate system at a target stitching level, a grid coordinate corresponding to the at least one first pixel in the grid coordinate system at the target stitching level according to the longitude and latitude parameters of the at least one first pixel, and to determine, based on the longitude and latitude parameters of the at least one second pixel, a grid coordinate corresponding to the at least one second pixel in the grid coordinate system; the target stitching level is positively correlated with the clarity of the stitched image; an overlapping pixel determining unit, configured to determine overlapping pixels among the at least one first pixel and the at least one second pixel according to the grid coordinates corresponding to the at least one first pixel and the grid coordinates corresponding to the at least one second pixel; An image stitching unit is used to stitch the at least one first pixel and the at least one second pixel into the grid coordinate system according to the grid coordinates corresponding to the at least one first pixel, the grid coordinates corresponding to the at least one second pixel and the overlapping pixels, so as to obtain a target image of the target area at the target stitching level.

11. The device according to claim 10, characterized in that The overlapping pixel determining unit is configured to determine, among the at least one first pixel and the at least one second pixel, the first pixel and the second pixel that have the same corresponding grid coordinates as a group of overlapping pixels; The image stitching unit includes: a target pixel feature acquisition subunit, configured to obtain target pixel features of the group of overlapping pixels based on original pixel features of the group of overlapping pixels; and use the original pixel features of non-overlapping pixels as target pixel features of the non-overlapping pixels; The pixel stitching subunit is used to stitch the target pixel features of the at least one first pixel and the target pixel features of the at least one second pixel into the grid coordinate system according to the positions of the grid coordinates corresponding to the pixels in the grid coordinate system, so as to obtain the target image of the target area at the target stitching level.

12. The device according to claim 10, characterized in that The grid coordinate system is specifically a tile map coordinate system; the tile map coordinate system includes tiles; the number of tiles is positively correlated with the target splicing level; The mapping relationship between the longitude and latitude and the grid coordinate system at the target splicing level includes: a first mapping relationship and a second mapping relationship; the first mapping relationship includes: the number of grids corresponding to 1 degree in the tile map coordinate system at the target splicing level; the second mapping relationship includes: the number of grids corresponding to 1 radian in the tile map coordinate system at the target splicing level; The pixel and grid mapping unit includes: a grid coordinate first component acquisition subunit, configured to obtain the longitude grid number of the target pixel in the tile map coordinate system according to the longitude parameter of the target pixel, the first mapping relationship, and the total number of grids in the longitude direction of the tile map coordinate system; The grid coordinate second component acquisition subunit is used to obtain the latitude grid ordinal number of the target pixel in the tile map coordinate system based on the latitude parameter of the target pixel, the second mapping relationship and the total number of grids in the latitude direction of the tile map coordinate system; the target pixel is any one of the at least one first pixel and the at least one second pixel.

13. The device according to claim 12, characterized in that The overlapping pixel determining unit includes: an operator unit, configured to obtain a first quotient and a first remainder of the longitude direction grid number of the target pixel and the longitude direction grid number, and obtain a second quotient and a second remainder of the latitude direction grid number of the target pixel and the latitude direction grid number; a tile identification obtaining subunit, configured to obtain an identification of a tile that matches the target pixel in the tile map coordinate system from the first quotient value and the second quotient value; an intra-tile coordinate acquisition subunit, configured to obtain the intra-tile coordinates of the target pixel matched in the tile map coordinate system from the first remainder and the second remainder; The overlapping pixel determination subunit is configured to treat the first pixel and the second pixel having the same identifier of the matched tile and the same coordinate within the tile as a group of overlapping pixels.

14. A computer device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the image stitching method according to any one of claims 1 to 9 according to instructions in the program code.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the image stitching method according to any one of claims 1 to 9.

Images