Landscape map updating method and system based on real scene image sharing and storage medium

Abstract

The application relates to the field of image data processing, in particular to a garden map updating method and system based on real scene image sharing and a storage medium, landscape element classification and space correlation of air-to-ground images of a garden space are carried out, a plurality of garden pattern distribution layers are generated, and an expected updating area is calibrated according to picture disturbance features of the layers; a plurality of real scene image libraries are formed by preprocessing real scene images, real scene images of the libraries are screened and fused, and optimized real scene images are obtained; the optimized real scene images are filled and fused with the expected updating area, all garden pattern distribution layers after the filling and fusion are superimposed, and a garden map is obtained. The application can implement detailed updating of different landscape elements in the garden space combined with real scene images, and can perform local fine real scene image fusion on the garden map, so that the garden map and the actual situation of the garden are matched and consistent, and the information of the map is refined.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image data processing, in particular to a garden map updating method and system based on real scene image sharing and a storage medium. BACKGROUND

[0002] As a landscape collection combining vegetation, water bodies, artificial buildings and other elements, the garden needs to be visually represented as a whole in daily maintenance and tour route setting, so that maintenance personnel and tourists can have a global understanding of the internal situation of the garden. At present, aerial photography equipment such as unmanned aerial vehicles is used to take aerial photographs of the garden, generate panoramic aerial photographs of the garden, and then convert the panoramic aerial photographs to obtain a garden map. Considering that aerial photography is affected by factors such as shooting resolution and shooting azimuth angle, the generated garden map cannot accurately reflect the detailed information of different elements inside the garden, and cannot meet the needs of fine and localized map queries of the garden.

[0003] In order to update the map data in a timely manner according to the real scene changes and increase the detailed information of the map, the prior art CN113538672A discloses an indoor electronic map updating method, which generates a local semantic map according to a plurality of user collected real scene images and reported data, compares the local semantic map with an existing indoor electronic map, timely discovers and extracts information related to changes in indoor map data, and realizes timely incremental update of indoor map change information. However, the above-mentioned invention patent is for updating map data in an indoor environment. Considering the layout rules of objects in the indoor environment, the shapes of the objects will not change, and the objects will not be disturbed by factors such as light and atmosphere. In the comparison process between the local semantic map and the indoor electronic map, only the changes in the shape outlines of the objects need to be concerned, and the differences between different types of objects do not need to be concerned. The above-mentioned method is not applicable to the garden scene which is complex and changeable in the internal and external environment. Therefore, how to update the generation of the garden map according to the layout of different types of landscape elements in the garden space, combined with the state of the landscape elements of the real scene image, has great significance for realizing the matching consistency of the map and the real situation of the garden and improving the fine information of the map. SUMMARY

[0004] In order to implement fine updating of different landscape elements in the garden space combined with real scene images, and to implement local fine real scene image fusion of the garden map, realize the matching consistency of the map and the real situation of the garden and improve the fine information of the map, the present application provides a garden map updating method based on real scene image sharing, which comprises the following steps:

[0005] S100: identify the landscape elements of the space-to-ground image of the garden, classify and space correlate the landscape elements, and generate a plurality of garden pattern distribution layers; obtain the picture disturbance features of each garden pattern distribution layer, and use the picture disturbance features to mark the expected update area of each garden pattern distribution layer;

[0006] S200: pre-process the real scene image of the shared garden space in the outside world, form a plurality of real scene image libraries, and determine the mapping relationship between the expected update area and the real scene image library; screen and fuse the real scene image of the real scene image library to obtain an optimized real scene image;

[0007] S300: according to the mapping relationship, fill and fuse the optimized real scene image with the corresponding expected update area; superimpose all garden pattern distribution layers that complete the expected update area filling and fusion to obtain a garden map of the garden space.

[0008] Preferably, in S100, the landscape elements of the space-to-ground image of the garden space are identified, the landscape elements are classified and associated with the space, and a plurality of garden pattern distribution layers are generated, specifically:

[0009] Obtain the space-to-ground image of the garden space at a plurality of time points respectively, perform multi-scale information capture on each space-to-ground image to obtain the shallow features and deep features of each space-to-ground image; select the space-to-ground image with the highest target segmentation quality according to the shallow features and deep features;

[0010] Perform target segmentation on the selected space-to-ground image to extract the landscape elements and their spatial features of the space-to-ground image; wherein the landscape elements include vegetation elements, water elements, and building elements; the spatial features include spatial position and spatial physical boundary; aggregate map the spatial features of each type of landscape element in the same spatial dimension to generate a plurality of garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

[0011] Preferably, in S100, the picture disturbance features of each garden pattern distribution layer are obtained to mark the expected update area of each garden pattern distribution layer, specifically:

[0012] Each garden pattern distribution layer is processed by cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer; according to the redundant information distribution, the detail coverage ratio features of the horizontal and vertical dimensions of each garden pattern distribution layer are determined to obtain the picture disturbance features;

[0013] The detail coverage ratio features of each garden pattern distribution layer are compared with the threshold value of each picture grid sub-area to mark the expected update area of each garden pattern distribution layer.

[0014] Preferably, in S200, the real scene images of the shared garden space outside are preprocessed to form a plurality of real scene image libraries, and a mapping relationship between the expected update area and the real scene image library is determined; the real scene images of the real scene image library are screened and fused to obtain an optimized real scene image, specifically as follows:

[0015] The real scene images of the shared garden space outside are collected, and the collected all real scene images are subjected to picture feature recognition and screening to retain part of the real scene images; wherein, the picture feature includes picture pixel parameter feature and picture element state feature;

[0016] The spatial orientation of each retained real scene image is identified, the spatial orientation correlation attribute between different retained real scene images is determined, all retained real scene images are distinguished, and a plurality of real scene image libraries are formed; the main spatial orientation of each real scene image library is compared with the distribution spatial orientation of the expected update area in the garden space to determine the mapping relationship between the expected update area and the real scene image library;

[0017] According to the landscape element detail information of each real scene image in the real scene image library, a plurality of real scene images without landscape element detail overlap are screened; the screened real scene images are subjected to a plurality of type landscape element extraction and fusion, and the real scene images are recombined in the picture to obtain an optimized real scene image.

[0018] Preferably, in S300, according to the mapping relationship, the optimized real scene image is filled and fused with the corresponding expected update area; all garden pattern distribution maps that have completed the expected update area filling and fusion are superimposed to obtain a garden map of the garden space, specifically as follows:

[0019] According to the mapping relationship, the optimized real scene image of the matched real scene image library is selected; the multi-scale spatial information of the background and landscape elements in the selected optimized real scene image is obtained, the multi-scale spatial information is refined to obtain embedded picture elements, and the embedded picture elements are filled and fused into the corresponding expected update area;

[0020] According to the boundary position of the garden space, all garden pattern distribution maps that have completed the expected update area filling and fusion are aligned and superimposed to obtain a garden map of the garden space.

[0021] On the other hand, the application provides a garden map update system based on real scene image sharing, the system comprising the following modules:

[0022] The layer generation module is used for identifying the landscape elements of the space-to-ground image of the garden space, classifying and spatially correlating the landscape elements, and generating a plurality of garden pattern distribution layers;

[0023] an update region calibration module, configured to obtain a picture disturbance feature of each garden pattern distribution layer, so as to calibrate a desired update region of each garden pattern distribution layer;

[0024] a gallery processing module, configured to pre-process real scene images of a garden space shared by an outside world, form a plurality of real scene galleries, and determine a mapping relationship between the desired update region and the real scene galleries;

[0025] an image optimization module, configured to screen and fuse real scene images of the real scene galleries to obtain optimized real scene images;

[0026] a filling fusion module, configured to fill and fuse the optimized real scene images and corresponding desired update regions according to the mapping relationship;

[0027] a layer superposition module, configured to superimpose all garden pattern distribution layers that have completed the filling fusion of the desired update regions to obtain a garden map of the garden space.

[0028] Preferably, the layer generation module is configured to identify landscape elements of the space-to-ground image of the garden space, classify the landscape elements in association with the space, and generate a plurality of garden pattern distribution layers, specifically:

[0029] obtain space-to-ground images of the garden space at a plurality of time points, perform multi-scale information capture on each space-to-ground image to obtain shallow features and deep features of each space-to-ground image, and select a space-to-ground image with the highest target segmentation quality according to the shallow features and the deep features;

[0030] perform target segmentation on the selected space-to-ground image to extract landscape elements and spatial features of the space-to-ground image; wherein the landscape elements include vegetation elements, water elements and building elements; the spatial features include spatial positions and spatial physical boundaries; perform aggregation mapping of the spatial features of each type of landscape element in the same spatial dimension to generate a plurality of garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element;

[0031] The update region calibration module is configured to obtain a picture disturbance feature of each garden pattern distribution layer, so as to calibrate a desired update region of each garden pattern distribution layer, specifically:

[0032] perform cross-attention fusion mechanism processing on each garden pattern distribution layer to obtain redundant information distribution of a horizontal dimension and a vertical dimension of a picture of each garden pattern distribution layer; determine a detail coverage ratio feature of the horizontal dimension and the vertical dimension of the picture of each garden pattern distribution layer according to the redundant information distribution, so as to obtain the picture disturbance feature;

[0033] The threshold comparison of each picture grid area is performed according to the detail coverage proportion characteristics of each garden pattern distribution layer, and the expected update area of each garden pattern distribution layer is calibrated.

[0034] Preferably, the gallery processing module is used for pre-processing the real scene images of the shared garden space in the outside world, forming a plurality of real scene galleries, and determining the mapping relationship between the expected update area and the real scene gallery, specifically:

[0035] Collect the real scene images of the shared garden space in the outside world, and perform picture feature recognition and screening on all collected real scene images, and retain part of the real scene images; wherein, the picture features include picture pixel parameter features and picture element state features;

[0036] The spatial orientation of each retained real scene image is identified, and the spatial orientation correlation attribute between different retained real scene images is determined, so as to distinguish all retained real scene images, form a plurality of real scene galleries, and compare the main spatial orientation of each real scene gallery with the distribution spatial orientation of the expected update area in the garden space, to determine the mapping relationship between the expected update area and the real scene gallery;

[0037] The image optimization module is used for screening and fusing the real scene images of the real scene gallery to obtain an optimized real scene image, specifically:

[0038] According to the landscape element detail information of each real scene image in the real scene gallery, a plurality of real scene images without landscape element detail overlap are screened; the screened real scene images are extracted and fused with a plurality of types of landscape elements, and the landscape elements are reorganized in the picture to obtain an optimized real scene image.

[0039] Preferably, the filling and fusing module is used for filling and fusing the optimized real scene image and the corresponding expected update area according to the mapping relationship, specifically:

[0040] According to the mapping relationship, the optimized real scene image of the matching real scene gallery is selected; the multi-scale spatial information of the background and the landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, so that the embedded picture elements are filled and fused into the corresponding expected update area;

[0041] The layer superposition module is used for superimposing all garden pattern distribution layers that complete the filling and fusion of the expected update area to obtain a garden map of the garden space, specifically:

[0042] According to the boundary position of the garden space, all garden pattern distribution layers that complete the filling and fusion of the expected update area are aligned and superimposed to obtain a garden map of the garden space.

[0043] In addition, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the method as described above when executed by a processor.

[0044] Compared with the prior art, the application has the following beneficial effects:

[0045] The landscape elements of the space-to-ground image of the garden space are identified, the landscape elements are classified and associated with the space, and a plurality of garden pattern distribution layers are generated; picture disturbance features of each garden pattern distribution layer are obtained to mark the expected update area of each garden pattern distribution layer. In the above manner, the vegetation, water body and buildings and other landscape elements of the garden space are separately and independently identified, and the garden pattern distribution layers corresponding to each type of landscape element are generated, so that the landscape elements of the garden space image are separated, which facilitates accurate marking of the expected update area of each layer from the detailed information of each type of landscape element itself, and ensures accurate adjustment and update of all types of landscape elements.

[0046] The real scene images of the garden space shared by the outside world are preprocessed to form a plurality of real scene image libraries, and the mapping relationship between the expected update area and the real scene image library is determined; the real scene images of the real scene image library are screened and fused to obtain an optimized real scene image. In the above manner, the real scene images are screened and classified, and a plurality of real scene images about the same or similar local area of the garden space are divided into the same real scene image library, so that the real scene image library provides sufficient real scene information about the local area of the garden space. The real scene images in the real scene image library are also optimally fused and reorganized to ensure that all landscape elements in the optimized real scene image are in the optimal visual viewing state.

[0047] According to the mapping relationship, the optimized real scene image is filled and fused with the corresponding expected update area; all garden pattern distribution layers in which the expected update area is filled and fused are superimposed to obtain a garden map of the garden space. In the above manner, the optimized real scene image is used to replace the expected update area, so that the landscape elements in the expected update area have more detailed information; the vegetation, water body and buildings in the garden space are also re-integrated into the same map space to obtain the garden map of the garden space, so that the map and the actual situation of the garden are matched and consistent, and the map information is refined. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without any creative effort. Among them:

[0049] Figure 1is a flow chart of the garden map updating method based on real scene image sharing provided by the present application.

[0050] Figure 2 is the air-to-ground image of the garden space.

[0051] Figure 3 is the shallow feature and the deep feature of the air-to-ground image

[0052] Figure 4 is the plurality of garden pattern distribution layers.

[0053] Figure 5 is the cross-attention fusion mechanism processing process.

[0054] Figure 6 is a structure diagram of the garden map updating system based on real scene image sharing provided by the present application. DETAILED DESCRIPTION

[0055] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, in order to facilitate the description, only the parts related to the present application are shown in the drawings, not all the structures. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] The terms "comprising" and "having" and any variations thereof in the present application are intended to cover non-exclusive inclusion. For example, the process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0057] In this paper, the phrase "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0058] Please refer to Figure 1 The present application provides a garden map updating method based on real scene image sharing, as shown in the figure, which comprises the following steps:

[0059] S100: Identify the landscape elements of the space-to-ground image of the garden space, classify and space correlate the landscape elements, generate multiple garden pattern distribution layers; obtain the picture disturbance characteristics of each garden pattern distribution layer, and mark the expected update area of each garden pattern distribution layer.

[0060] Further, in S100, the landscape elements of the space-to-ground image of the garden space are identified, the landscape elements are classified and space correlated, and multiple garden pattern distribution layers are generated, specifically:

[0061] Obtain the space-to-ground image of the garden space at several time points, capture multi-scale information for each space-to-ground image, and obtain the shallow features and deep features of each space-to-ground image; select the space-to-ground image with the highest target segmentation quality according to the shallow features and deep features;

[0062] Perform target segmentation on the selected space-to-ground image to extract the landscape elements and their spatial features; wherein the landscape elements include vegetation elements, water elements, and building elements; the spatial features include spatial position and spatial physical boundary; aggregate map the spatial features of each type of landscape element in the same spatial dimension to generate multiple garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

[0063] The garden space can be, but is not limited to, a park, a wilderness forest, and other three-dimensional spaces that collect plants, lakes, and artificial buildings. The garden space has the characteristics of a large area and a three-dimensional staggered structure. In order to reduce the image acquisition efficiency and comprehensiveness of the garden space, aerial photography equipment such as a drone can be used to patrol and photograph the garden space to obtain a space-to-ground image of the entire range of the garden space. Please refer to Figure 2 For a space-to-ground image of a municipal park, the above space-to-ground image includes trees, lawns, and other vegetation, artificial lakes, rivers, and other water bodies, pavilions, steps, and other artificial buildings. Vegetation, water, and artificial buildings have different forms; among them, vegetation is in a growing state, its own shape changes, and it continuously changes dynamically under external light and atmospheric flow; water itself has flowability and also continuously changes dynamically under external light and atmospheric flow; artificial buildings are in a static state compared to vegetation and water, and the construction lines and contours of the buildings do not change basically. Through the above analysis, it can be known that the landscape elements of the garden space, such as vegetation, water, and artificial buildings, have great differences in spatial distribution and form, and the change speed and degree of different landscape elements are also different. If all types of landscape elements in the entire range of the garden space are updated uniformly, not only a large amount of real scene data is needed, but also a large amount of computing power and memory resources are consumed, which cannot achieve rapid and accurate updating of the garden map. Therefore, the garden map can be updated independently for the three types of landscape elements of vegetation, water, and artificial buildings.

[0064] Considering the potential for significant internal and external interference during a single aerial inspection of the garden space, to minimize the impact of interference, it is necessary to take aerial photos of the garden space at several time points to obtain multiple air-to-ground images. A multi-scale feature extraction network based on the Transformer architecture, such as the Adaptive Subspace Feature Fusion module, is employed to capture multi-scale information for each air-to-ground image, obtaining shallow and deep features for each image. Shallow features refer to spatial details such as the boundary contours of targets within the image, while deep features refer to the specific semantic information of the image. (See also...) Figure 3 It utilizes the adaptive subspace feature fusion module to... Figure 2 Shallow features obtained by capturing multi-scale information from multiple regions within an air-to-ground image (corresponding to...) Figure 3 (a) and deep features (corresponding to) Figure 3 (b) realizes the multi-scale feature information representation of air-to-ground images. Then, based on the information content of all shallow features and all deep features under each air-to-ground image, the total amount of feature information corresponding to each air-to-ground image is determined, and the air-to-ground image with the highest total amount of feature information is determined as the air-to-ground image with the highest target segmentation quality, ensuring that the selected air-to-ground image has enough detailed information to satisfy target segmentation.

[0065] The U-Net segmentation network can be used to segment selected air-to-ground images, extracting the spatial locations and physical boundaries of vegetation, water, and building elements. This allows for the separate segmentation and extraction of these three types of landscape elements within the air-to-ground image. Furthermore, the U-Net fusion network can be used to aggregate and map all elements and their spatial features under each type of landscape element along the same spatial dimension, generating multiple garden pattern distribution layers. Specifically, the U-Net fusion network aggregates all vegetation elements and their spatial features, all water elements and their spatial features, and all building elements and their spatial features along the same spatial coordinate dimension, generating three garden pattern distribution layers corresponding to vegetation, water, and building elements, respectively. Please refer to [link / reference]. Figure 4 (a), (b), and (c) in the text represent the garden pattern distribution layers for the corresponding vegetation elements, water elements, and building elements, respectively. Each garden pattern distribution layer represents the spatial distribution density of vegetation elements, water elements, and building elements within the global grid of the garden space. Among them, the darker the color of the grid in the garden pattern distribution layer, the greater the spatial distribution density of vegetation elements, water elements, and building elements.

[0066] Further, in S100, the picture disturbance features of each garden pattern distribution layer are obtained to demarcate the expected update area of each garden pattern distribution layer, specifically:

[0067] Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer; and the picture disturbance features are obtained according to the detail coverage proportion features of the horizontal and vertical dimensions of each garden pattern distribution layer.

[0068] The detail coverage proportion features of each garden pattern distribution layer are compared with the threshold value of each picture grid sub-area to demarcate the expected update area of each garden pattern distribution layer.

[0069] Referring to Figure 5 The cross-attention fusion mechanism network can be used to distribute the redundant information such as blank information and / or noise information in the horizontal and vertical dimensions of each garden pattern distribution layer in the entire picture range of the layer. It can be understood that the entire picture of the garden pattern distribution layer contains effective target information (such as the edge profile, chrominance, luminance, etc. of the target), blank information and noise information. In actual image update processing, only the above effective target information is concerned, while the blank information and noise information do not contribute to the image update processing, but rather cover the effective target information. The more blank information and noise information in a grid sub-area of the layer picture, the higher the degree of covering the effective target information in the above grid sub-area. According to the redundant information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer, the detail information coverage proportion value of the target edge profile, chrominance, luminance, etc. in the horizontal and vertical dimensions of each garden pattern distribution layer is determined as the picture disturbance feature. Those skilled in the art can process the garden pattern distribution layer by the cross-attention fusion mechanism under the Python framework, which will not be described in detail here.

[0070] The detail information coverage proportion value of each picture grid sub-area in each garden pattern distribution layer is also compared with the threshold value. If the detail information coverage proportion value exceeds the preset proportion threshold value, the corresponding picture grid sub-area is determined as a detail unclear grid sub-area; otherwise, the corresponding picture grid sub-area is not determined as a detail unclear grid sub-area; and the area formed by connecting all the detail unclear grid sub-areas is demarcated as the expected update area of the garden pattern distribution layer, which limits the area range for updating the layer content by using real scene images in the subsequent process.

[0071] S200: preprocessing the real scene images of the garden space shared by the outside world, forming a plurality of real scene image libraries, and determining the mapping relationship between the expected update area and the real scene image library; screening and fusing the real scene images of the real scene image library to obtain optimized real scene images.

[0072] Further, in S200, the real scene images of the garden space shared by the outside world are preprocessed to form a plurality of real scene image libraries, and the mapping relationship between the expected update area and the real scene image library is determined; the real scene images of the real scene image library are screened and fused to obtain optimized real scene images, specifically:

[0073] Collecting real scene images of the garden space shared by the outside world, performing picture feature recognition and screening on all collected real scene images, and retaining part of the real scene images; wherein the picture features include picture pixel parameter features and picture element state features;

[0074] Performing spatial orientation recognition on each retained real scene image, determining the spatial orientation correlation attributes between different retained real scene images, and distinguishing all retained real scene images in this way to form a plurality of real scene image libraries; comparing the main spatial orientation of each real scene image library with the distribution spatial orientation of the expected update area in the garden space to determine the mapping relationship between the expected update area and the real scene image library;

[0075] According to the landscape element detail information of each real scene image in the real scene image library, a plurality of real scene images without landscape element detail overlap are screened; the screened real scene images are subjected to a plurality of type landscape element extraction and fusion, and landscape element reorganization in the picture to obtain optimized real scene images.

[0076] The real scene images can be, but are not limited to, images captured by garden maintenance personnel using terminals such as smart phones in the real environment of the garden space, and each real scene image corresponds to a real scene image of a local area in the garden space. By collecting all real scene images of the garden space shared by the outside world, picture recognition is performed on each real scene image to obtain picture pixel resolution, picture pixel contrast and other pixel parameter features of each real scene image, as well as the number of landscape elements in the picture, the completeness of the landscape elements (i.e. the ratio between the display part of the landscape elements in the picture and the whole part of the landscape elements), and other element state features. According to the above picture pixel parameter features and picture element state features, all real scene images are screened, and real scene images with picture pixel resolution less than a preset resolution threshold, or picture pixel contrast less than a preset contrast threshold, or the number of landscape elements in the picture less than a preset number threshold, or the completeness of the landscape elements less than a preset degree threshold are removed, thereby retaining part of the real scene images.

[0077] The spatial orientation recognition is performed on each reserved real scene image again, and a spatial orientation overlap angle value between any two reserved real scene images is determined, which is used as a spatial orientation correlation attribute between different reserved real scene images. If the spatial orientation overlap angle value between the two reserved real scene images is greater than or equal to a preset angle threshold, the two reserved real scene images are divided into the same real scene image library; otherwise, the two reserved real scene images are divided into two different real scene image libraries. In the above manner, all the reserved real scene images are distinguished, and a plurality of real scene image libraries are formed. All the real scene images under each real scene image library can be understood as corresponding to the same spatial orientation of the garden space, and the overlap angle interval of the spatial orientation range of all the real scene images under each real scene image library is determined as the main spatial orientation of each real scene image library. Through the above analysis, it can be known that the main spatial orientation of the real scene image library represents the local area of the garden space that can be captured by all the real scene images in the real scene image library. Therefore, the main spatial orientation of the real scene image library is compared with the distribution spatial orientation of the expected update region in the garden space. If the main spatial orientation is consistent with the distribution spatial orientation, a mapping relationship between the real scene image library and the expected update region is determined, which facilitates the subsequent accurate selection of a real scene image library from the plurality of real scene image libraries to provide landscape element data for filling and fusing the expected update region.

[0078] In addition, for all the real scene images in each real scene image library, the edge contour and other detail information of the landscape elements of each real scene image are obtained, and the edge contour and other detail information of the landscape elements of all the real scene images are compared and screened to obtain a plurality of real scene images without overlapping of the detail information of the landscape elements. That is, for a plurality of real scene images with overlapping of the edge contour and other detail information of the landscape elements, only one real scene image is reserved. The plurality of real scene images thus reserved do not have overlapping of the edge contour and other detail information of the landscape elements. Then, the U-Net fusion network is used to extract and fuse the three types of landscape elements, i.e., vegetation, water body, and building, of the real scene images thus screened and reserved, and to reorganize them in the same picture space to obtain optimized real scene images. In the above manner, the landscape elements with the best quality in all the real scene images in the same real scene image library are fused and reorganized to ensure that all the landscape elements in the optimized real scene images are in the best visual viewing state.

[0079] S300: According to the mapping relationship, the optimized real scene image is filled and fused with the corresponding expected update region; and all the garden pattern distribution maps that have completed the filling and fusion of the expected update region are superimposed to obtain a garden map of the garden space.

[0080] In S300, according to the mapping relationship, the optimized real scene image is filled and fused with the corresponding expected update region; and all the garden pattern distribution maps that have completed the filling and fusion of the expected update region are superimposed to obtain a garden map of the garden space. Specifically,

[0081] According to the mapping relationship, an optimized real scene image matching the real scene image library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and embedded picture elements are obtained by refining the multi-scale spatial information, so that the embedded picture elements are filled and fused into the corresponding expected update area;

[0082] According to the boundary position of the garden space, all garden pattern distribution layers that complete the filling and fusion of the expected update area are aligned and superimposed to obtain a garden map of the garden space.

[0083] According to the mapping relationship, an optimized real scene image matching the real scene image library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and embedded picture elements are obtained by refining the multi-scale spatial information, so that the embedded picture elements are filled and fused into the corresponding expected update area;

[0084] Please refer to Figure 6 The present application provides a garden map updating system based on real scene image sharing, which comprises the following modules:

[0085] A layer generation module is used to identify the landscape elements of the space-to-ground image of the garden space, classify and associate the landscape elements with the space, and generate a plurality of garden pattern distribution layers;

[0086] An update area calibration module is used to obtain the picture disturbance features of each garden pattern distribution layer, so as to calibrate the expected update area of each garden pattern distribution layer;

[0087] A library processing module is used to pre-process the real scene images of the garden space shared by the outside world, form a plurality of real scene image libraries, and determine the mapping relationship between the expected update area and the real scene image library;

[0088] An image optimization module is used to screen and fuse the real scene images of the real scene image library to obtain an optimized real scene image;

[0089] A filling and fusion module is used to fill and fuse the optimized real scene image and the corresponding expected update area according to the mapping relationship.

[0090] A layer superimposition module is configured to superimpose all the garden pattern distribution layers to obtain a garden map of the garden space.

[0091] Further, the layer generation module is configured to identify landscape elements of the space-to-ground image of the garden space, classify the landscape elements in association with the space, and generate a plurality of garden pattern distribution layers, specifically:

[0092] The space-to-ground images of the garden space at a plurality of time points are obtained, multi-scale information of each space-to-ground image is captured to obtain shallow features and deep features of each space-to-ground image, and a space-to-ground image with the highest target segmentation quality is selected according to the shallow features and the deep features.

[0093] Target segmentation is performed on the selected space-to-ground image to extract landscape elements and spatial features of the space-to-ground image, wherein the landscape elements include vegetation elements, water elements, and building elements, and the spatial features include spatial positions and spatial physical boundaries; the spatial features of each type of landscape element are aggregated and mapped in the same spatial dimension to generate a plurality of garden pattern distribution layers; and each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

[0094] The update region calibration module is configured to obtain picture disturbance features of each garden pattern distribution layer to calibrate desired update regions of each garden pattern distribution layer, specifically:

[0095] Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain redundant information distribution of the horizontal dimension and the vertical dimension of the picture of each garden pattern distribution layer; the detail coverage ratio features of the horizontal dimension and the vertical dimension of the picture of each garden pattern distribution layer are determined according to the redundant information distribution to obtain the picture disturbance features.

[0096] The detail coverage ratio features of each garden pattern distribution layer are compared with threshold values of each picture grid sub-region to calibrate the desired update regions of each garden pattern distribution layer.

[0097] Further, the gallery processing module is configured to preprocess real scene images of the garden space shared by the outside world to form a plurality of real scene galleries and determine a mapping relationship between the desired update regions and the real scene galleries, specifically:

[0098] The real scene images of the garden space shared by the outside world are collected, and picture feature recognition and screening are performed on all the collected real scene images to retain part of the real scene images; wherein the picture features include picture pixel parameter features and picture element state features.

[0099] The spatial orientation of each reserved real scene image is recognized, the spatial orientation correlation attribute between different reserved real scene images is determined, all the reserved real scene images are distinguished, and a plurality of real scene image libraries are formed; the main spatial orientation of each real scene image library is compared with the distribution spatial orientation of the expected update region in the garden space, and the mapping relationship between the expected update region and the real scene image library is determined;

[0100] The image optimization module is used for screening and fusing the real scene images of the real scene image library to obtain optimized real scene images, specifically as follows:

[0101] According to the landscape element detail information of each real scene image in the real scene image library, a plurality of real scene images without landscape element detail overlap are screened; the screened real scene images are subjected to a plurality of type landscape element extraction and fusion and a landscape element in-picture reorganization to obtain optimized real scene images.

[0102] Further, the filling and fusing module is used for filling and fusing the optimized real scene images and the corresponding expected update region according to the mapping relationship, specifically as follows:

[0103] According to the mapping relationship, the optimized real scene images of the matched real scene image library are selected; the multi-scale spatial information of the background and the landscape elements in the selected optimized real scene images is obtained, and the embedded picture elements are obtained by refining the multi-scale spatial information, so that the embedded picture elements are filled and fused into the corresponding expected update region;

[0104] The layer superposition module is used for superimposing all the garden pattern distribution layers that have completed the filling and fusing of the expected update region to obtain a garden map of the garden space, specifically as follows:

[0105] According to the boundary position of the garden space, the garden pattern distribution layers that have completed the filling and fusing of the expected update region are aligned and superimposed to obtain a garden map of the garden space.

[0106] The operation and effect of the garden map updating system based on real scene image sharing of the present application are consistent with the above-mentioned garden map updating method based on real scene image sharing, and the garden map updating system based on real scene image sharing will not be repeated here.

[0107] In an embodiment of the present application, the present application further provides a computer readable storage medium, and a computer program is stored on the readable storage medium, and the computer program realizes the above-mentioned method when executed by a processor.

[0108] In an embodiment of the present application, the present application further provides a computer device, and the computer device at least includes a memory and a processor, and a computer program is stored on the memory, and the computer program realizes the above-mentioned method when executed by the processor.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of the necessary universal hardware platform, and of course can also be implemented by means of the combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of a computer product in essence or in the form of a contribution to the prior art. The present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0110] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and are not limited thereto, and other embodiments can also be used; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A garden map updating method based on real image sharing, characterized in that, The method comprises the following steps: S100: identifying landscape elements of the space-to-ground image of the garden space, classifying and associating the landscape elements with the space, and generating a plurality of garden pattern distribution layers; Obtain the picture disturbance feature of each garden pattern distribution layer, and use it to mark the expected update area of each garden pattern distribution layer, specifically: Perform cross-attention fusion mechanism processing on each garden pattern distribution layer to obtain the redundancy information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer; according to the redundancy information distribution, determine the detail coverage ratio feature of the horizontal and vertical dimensions of each garden pattern distribution layer, to obtain the picture disturbance feature; Compare the threshold value of each garden pattern distribution layer with the grid sub-area of the detail coverage ratio feature, and mark the expected update area of each garden pattern distribution layer; S200: preprocessing the real scene images of the garden space shared by the outside world to form a plurality of real scene image libraries, and determining the mapping relationship between the expected update area and the real scene image library; screening and fusing the real scene images of the real scene image library to obtain optimized real scene images, specifically: Collect the real scene images of the garden space shared by the outside world, and perform picture feature recognition and screening on all collected real scene images, and retain part of the real scene images; wherein the picture feature includes picture pixel parameter feature and picture element state feature; Identify the spatial orientation of each retained real scene image to determine the spatial orientation association attribute between different retained real scene images, thereby distinguishing all retained real scene images to form a plurality of real scene image libraries; compare the main spatial orientation of each real scene image library with the distribution spatial orientation of the expected update area in the garden space to determine the mapping relationship between the expected update area and the real scene image library; According to the landscape element detail information of each real scene image in the real scene image library, screen a plurality of real scene images without landscape element detail overlap; extract and fuse a plurality of types of landscape elements from the screened real scene images, and reorganize the landscape elements in the picture to obtain optimized real scene images; S300: according to the mapping relationship, fill and fuse the optimized real scene images with the corresponding expected update area; superimpose all garden pattern distribution layers that have completed the expected update area filling and fusion to obtain the garden map of the garden space.

2. The method of claim 1, wherein, In S100, the landscape elements of the space-to-ground image of the garden space are identified, the landscape elements are classified and associated with the space, and a plurality of garden pattern distribution layers are generated, specifically: Obtain the space-to-ground image of the garden space at a plurality of time points, capture multi-scale information from each space-to-ground image to obtain shallow features and deep features of each space-to-ground image; According to the shallow features and deep features, select the space-to-ground image with the highest target segmentation quality; The selected air-to-ground image is subjected to target segmentation, and landscape elements and their spatial features of the air-to-ground image are extracted; wherein the landscape elements include vegetation elements, water body elements, and building elements; and the spatial features include spatial positions and spatial physical boundaries; the spatial features of each type of landscape element are subjected to aggregation mapping in the same spatial dimension, and a plurality of garden pattern distribution layers are generated; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

3. The method of claim 1, wherein, In S300, according to the mapping relationship, the optimized real scene image is filled and fused with the corresponding expected update area; and all garden pattern distribution layers that have completed the expected update area filling and fusion are superimposed to obtain a garden map of the garden space, specifically as follows: According to the mapping relationship, an optimized real scene image matching the real scene image library is selected; multi-scale spatial information of the background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, so as to fill and fuse the embedded picture elements into the corresponding expected update area; According to the boundary position of the garden space, all garden pattern distribution layers that have completed the expected update area filling and fusion are aligned and superimposed to obtain a garden map of the garden space.

4. A garden map updating system based on real image sharing, characterized by, The system comprises the following modules: A layer generation module for identifying landscape elements of an air-to-ground image of a garden space, classifying and associating the landscape elements with spaces, and generating a plurality of garden pattern distribution layers; An update area calibration module for obtaining picture disturbance features of each garden pattern distribution layer, and calibrating the expected update area of each garden pattern distribution layer, specifically as follows: Each garden pattern distribution layer is subjected to cross-attention fusion mechanism processing to obtain redundant information distribution of the horizontal and vertical dimensions of the picture of each garden pattern distribution layer; according to the redundant information distribution, the detail coverage proportion features of the horizontal and vertical dimensions of the picture of each garden pattern distribution layer are determined to obtain the picture disturbance features; The detail coverage proportion features of each garden pattern distribution layer are subjected to threshold comparison of each picture grid sub-area to calibrate the expected update area of each garden pattern distribution layer; A library processing module for preprocessing real scene images of a garden space shared by the outside world, forming a plurality of real scene image libraries, and determining the mapping relationship between the expected update area and the real scene image library, specifically as follows: The real scene images of the garden space shared by the outside world are collected, and picture feature recognition and screening are performed on all the collected real scene images to retain part of the real scene images; wherein the picture features include picture pixel parameter features and picture element state features; The spatial orientation of each retained real scene image is identified, and the spatial orientation association attributes between different retained real scene images are determined to distinguish all the retained real scene images and form a plurality of real scene image libraries; the main spatial orientation of each real scene image library is compared with the distribution spatial orientation of the expected update area in the garden space to determine the mapping relationship between the expected update area and the real scene image library; An image optimization module is configured to screen and fuse the real scene images in the real scene image library to obtain optimized real scene images, specifically as follows: According to the landscape element detail information of each real scene image in the real scene image library, a plurality of real scene images without overlapping landscape element details are screened; the screened real scene images are subjected to a plurality of type landscape element extraction and fusion and a landscape element reorganization in a picture to obtain optimized real scene images; A filling and fusion module is configured to fill and fuse the optimized real scene images and corresponding expected update regions according to the mapping relationship; A layer superimposition module is configured to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update regions to obtain a garden map of the garden space.

5. The system of claim 4, wherein The layer generation module is configured to identify landscape elements of the space-to-ground image of the garden space, classify and space correlate the landscape elements, and generate a plurality of garden pattern distribution layers, specifically as follows: Obtain the space-to-ground images of the garden space at a plurality of time points, capture the shallow and deep features of each space-to-ground image through multi-scale information capture; Select the space-to-ground image with the highest target segmentation quality according to the shallow and deep features; Perform target segmentation on the selected space-to-ground image to extract the landscape elements and their spatial features of the space-to-ground image; wherein the landscape elements include vegetation elements, water elements, and building elements; the spatial features include spatial positions and spatial physical boundaries; aggregate and map the spatial features of each type of landscape element in the same spatial dimension to generate a plurality of garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

6. The system of claim 4, wherein The filling and fusion module is configured to fill and fuse the optimized real scene images and corresponding expected update regions according to the mapping relationship, specifically as follows: According to the mapping relationship, select the optimized real scene image of the real scene image library that matches; obtain the multi-scale spatial information of the background and landscape elements in the selected optimized real scene image, and refine the multi-scale spatial information to obtain embedded picture elements, so as to fill and fuse the embedded picture elements into the corresponding expected update region; The layer superimposition module is configured to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update regions to obtain a garden map of the garden space, specifically as follows: According to the boundary position of the garden space, align and superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update regions to obtain a garden map of the garden space.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the method of any one of claims 1-3.

Images