Garden map updating method and system based on live-action image sharing and storage medium

Abstract

The invention relates to the field of image data processing, in particular to a garden map updating method and system based on live-action image sharing and a storage medium, landscape elements of air-to-ground images of a garden space are classified and spatially associated, a plurality of garden pattern distribution map layers are generated, and an expected updating area is calibrated according to picture disturbance characteristics of the map layers; preprocessing the live-action image to form a plurality of live-action image libraries, and screening and fusing the live-action images of the image libraries to obtain an optimized live-action image; and filling and fusing the optimized live-action image and the expected updating area, and overlapping all the filled and fused garden pattern distribution diagrams to obtain a garden map. According to the method, detail updating combined with the real-scene image can be implemented on different landscape elements in the garden space, local refined real-scene image fusion can be performed on the garden map, matching consistency of the map and the garden field condition is realized, and map information refinement is improved.

Description

Technical Field

[0001] The present invention relates to the field of image data processing, and in particular to a garden map updating method, system and storage medium based on real-scene image sharing. Background Art

[0002] As a landscape that integrates diverse elements such as vegetation, water bodies, and artificial structures, gardens require a holistic visualization for routine maintenance and tour route planning. This provides both maintenance personnel and visitors with a comprehensive understanding of the garden's interior. Currently, aerial photography equipment such as drones is used to capture panoramic aerial images of gardens from aerial perspectives, which are then converted into garden maps. However, due to factors such as resolution and azimuth angle, these aerial images often lack the ability to accurately reflect the detailed information of different garden elements, making them inadequate for detailed and localized garden map queries.

[0003] To promptly update map data and add detailed map information based on real-world changes, prior art CN113538672A discloses a method for updating indoor electronic maps. This method generates a local semantic map based on real-world images and reported data collected by multiple users. The local semantic map is then compared with an existing indoor electronic map to promptly detect and extract information related to changes in indoor map data, enabling timely incremental updates of indoor map change information. However, this invention patent only updates map data for indoor environments. Considering the layout of objects in indoor environments, object shapes do not change, and objects are not affected by factors such as lighting and atmospheric conditions. The comparison between the local semantic map and the indoor electronic map only focuses on changes in object outlines, not on differences between different types of objects. This method is not suitable for garden scenes, where both internal and external environments are complex and constantly changing. Therefore, generating a garden map based on the layout of different types of landscape elements within a garden space, combined with the state of landscape elements in real-world images, is crucial for achieving consistent alignment between the map and the actual landscape and improving the refinement of map information. Summary of the Invention

[0004] In order to implement detailed updates of different landscape elements in a garden space by combining real-scene images, and to perform local refined real-scene image fusion on a garden map, thereby achieving consistency between the map and the actual situation of the garden and improving the refinement of map information, the present invention provides a garden map updating method based on real-scene image sharing, which includes the following steps:

[0005] S100: Identifying landscape elements of an air-to-ground image of a garden space, classifying and spatially associating the landscape elements, and generating multiple garden pattern distribution layers; obtaining image disturbance features of each garden pattern distribution layer, and using them to calibrate a desired update area of each garden pattern distribution layer;

[0006] S200: Pre-processing the real-scene images of the garden space shared by the outside world to form multiple real-scene image libraries, and determining the mapping relationship between the desired update area and the real-scene image library; screening and fusing the real-scene images of the real-scene image library to obtain an optimized real-scene image;

[0007] S300: filling and fusing the optimized real scene image with the corresponding expected update area according to the mapping relationship; and superimposing all garden pattern distribution layers in which the expected update area is filled and fused to obtain a garden map of the garden space.

[0008] Preferably, in S100, the landscape elements of the air-to-ground image of the garden space are identified, the landscape elements are classified and spatially associated, and a plurality of garden pattern distribution layers are generated, specifically:

[0009] Acquire air-to-ground images of the garden space at several time points, perform multi-scale information capture on each air-to-ground image, and obtain shallow features and deep features of each air-to-ground image; select the air-to-ground image with the highest object segmentation quality based on the shallow features and deep features;

[0010] The selected air-to-ground image is segmented to extract the landscape elements and their spatial features; wherein the landscape elements include vegetation elements, water elements, and architectural elements; the spatial features include spatial position and spatial physical boundaries; the spatial features of each type of landscape element are aggregated and mapped in the same spatial dimension to generate multiple garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

[0011] Preferably, in S100, the image disturbance feature of each garden pattern distribution layer is obtained to calibrate the expected update area of each garden pattern distribution layer, specifically:

[0012] Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of the image of each garden pattern distribution layer; based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of the image of each garden pattern distribution layer are determined to obtain the image disturbance characteristics;

[0013] The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

[0014] Preferably, in S200, the real-scene images of the garden space shared by the outside world are pre-processed to form a plurality of real-scene libraries, and a mapping relationship between the desired update area and the real-scene library is determined; the real-scene images of the real-scene library are screened and integrated to obtain an optimized real-scene image, specifically:

[0015] Collecting real-life images of garden spaces uploaded and shared by the outside world, identifying and screening all collected real-life images, and retaining some real-life images; wherein the image features include image pixel parameter features and image element state features;

[0016] Perform spatial orientation recognition on each retained real-scene image, determine spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the desired update area in the garden space, and determine a mapping relationship between the desired update area and the real-scene library;

[0017] According to the landscape element detail information of each real scene image in the real scene library, a number of real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and merged from the screened real scene images, and the landscape elements are reorganized within the picture to obtain an optimized real scene image.

[0018] Preferably, in S300, according to the mapping relationship, the optimized real scene image is filled and fused with the corresponding desired update area; all garden pattern distribution layers that have completed the filling and fusion of the desired update area are superimposed to obtain a garden map of the garden space, specifically:

[0019] According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding desired update area;

[0020] According to the boundary position of the garden space, all garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain a garden map of the garden space.

[0021] In another aspect, the present invention provides a garden map updating system based on real-scene image sharing, the system comprising the following modules:

[0022] A layer generation module is used to identify landscape elements in the air-to-ground image of the garden space, classify the landscape elements and associate them with space, and generate multiple garden pattern distribution layers;

[0023] An update region calibration module is used to obtain the image disturbance characteristics of each garden pattern distribution layer and to calibrate the expected update region of each garden pattern distribution layer;

[0024] A picture library processing module is used to pre-process the real-scene images of the garden space shared by the outside world to form multiple real-scene picture libraries, and determine the mapping relationship between the desired update area and the real-scene picture library;

[0025] An image optimization module, configured to filter and fuse the real-scene images in the real-scene image library to obtain optimized real-scene images;

[0026] A filling and fusion module, configured to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship;

[0027] The layer superposition module is used to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update area to obtain a garden map of the garden space.

[0028] Preferably, the layer generation module is used to identify the landscape elements of the air-to-ground image of the garden space, classify the landscape elements and associate them with the space, and generate multiple garden pattern distribution layers, specifically:

[0029] Acquire air-to-ground images of the garden space at several time points, perform multi-scale information capture on each air-to-ground image, and obtain shallow features and deep features of each air-to-ground image; select the air-to-ground image with the highest object segmentation quality based on the shallow features and deep features;

[0030] Performing target segmentation on the selected air-to-ground image to extract landscape elements and their spatial features; wherein the landscape elements include vegetation elements, water elements, and architectural elements; and the spatial features include spatial location and spatial physical boundaries; performing aggregation mapping of the spatial features of each type of landscape element in the same spatial dimension to generate multiple garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element;

[0031] The update region calibration module is used to obtain the image disturbance features of each garden pattern distribution layer, and to calibrate the expected update region of each garden pattern distribution layer, specifically:

[0032] Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of the image of each garden pattern distribution layer; based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of the image of each garden pattern distribution layer are determined to obtain the image disturbance characteristics;

[0033] The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

[0034] Preferably, the image library processing module is used to pre-process the real-view images of the garden space shared by the outside world to form multiple real-view image libraries, and determine the mapping relationship between the expected update area and the real-view image library, specifically:

[0035] Collecting real-life images of garden spaces uploaded and shared by the outside world, identifying and screening all collected real-life images, and retaining some real-life images; wherein the image features include image pixel parameter features and image element state features;

[0036] Perform spatial orientation recognition on each retained real-scene image, determine spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the desired update area in the garden space, and determine a mapping relationship between the desired update area and the real-scene library;

[0037] The image optimization module is used to filter and fuse the real-scene images in the real-scene image library to obtain optimized real-scene images, specifically:

[0038] According to the landscape element detail information of each real scene image in the real scene library, a number of real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and merged from the screened real scene images, and the landscape elements are reorganized within the picture to obtain an optimized real scene image.

[0039] Preferably, the filling and fusion module is used to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship, specifically:

[0040] According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding desired update area;

[0041] The layer superposition module is used to superimpose all garden pattern distribution layers that have completed the filling and fusion of the desired update area to obtain a garden map of the garden space, specifically:

[0042] According to the boundary position of the garden space, all garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain a garden map of the garden space.

[0043] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] Identify landscape elements in an air-to-ground image of a garden space, classify these elements and spatially associate them to generate multiple garden pattern distribution layers. Obtain the image disturbance characteristics of each garden pattern distribution layer to calibrate the desired update area for each garden pattern distribution layer. This method allows for the independent identification of landscape elements such as vegetation, water bodies, and buildings within the garden space, generating a garden pattern distribution layer corresponding to each type of landscape element. This allows for the separation of landscape elements within the garden space image. This facilitates accurate calibration of the desired update area for each layer based on the detailed information of each type of landscape element, ensuring accurate adjustment and updates for all types of landscape elements.

[0046] Preprocessing the shared real-life images of the garden space to form multiple real-life image libraries, and determining the mapping relationship between the desired update areas and the real-life image libraries; then filtering and integrating the real-life images in the real-life image libraries to obtain optimized real-life images. This method filters and classifies the real-life images, grouping multiple real-life images of the same or similar local areas of the garden space into the same real-life image library, ensuring that the real-life image library provides sufficient real-life information on landscape elements within the local areas of the garden space. Furthermore, the real-life images within the real-life image library are integrated and reorganized to obtain the highest-quality landscape elements, ensuring that all landscape elements within the optimized real-life image are in optimal visual condition.

[0047] Based on the mapping relationship, the optimized real-world image is infilled and fused with the corresponding desired update area. All garden pattern distribution layers that have completed the infill fusion of the desired update area are superimposed to obtain a garden map of the garden space. By replacing the desired update area with the optimized real-world image, the landscape elements within the desired update area are enriched with more detailed information. The three types of elements within the garden space—vegetation, water bodies, and buildings—are also reintegrated into the same map space to obtain a garden map of the forest space, achieving consistency between the map and the actual garden conditions and improving the refinement of the map information. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. Among them:

[0049] Figure 1The present invention provides a flow chart of a garden map updating method based on real-scene image sharing.

[0050] Figure 2 It is an air-to-ground image of the garden space.

[0051] Figure 3 are the shallow features and deep features of air-to-ground images

[0052] Figure 4 It is a layer of multiple garden pattern distribution.

[0053] Figure 5 It is the processing process of the cross-attention fusion mechanism.

[0054] Figure 6 This is a structural diagram of a garden map updating system based on real-scene image sharing provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It will be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] The terms "comprise," "comprising," and "having," and any variations thereof, as used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0057] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] See also Figure 1 As shown, the present invention provides a garden map updating method based on real-scene image sharing, which includes the following steps:

[0059] S100: Identify landscape elements of an air-to-ground image of a garden space, classify the landscape elements and spatially associate them to generate multiple garden pattern distribution layers; obtain image disturbance features of each garden pattern distribution layer, and use them to calibrate the expected update area of each garden pattern distribution layer.

[0060] Furthermore, in S100, the landscape elements of the air-to-ground image of the garden space are identified, the landscape elements are classified and spatially associated, and a plurality of garden pattern distribution layers are generated, specifically:

[0061] Acquire air-to-ground images of the garden space at several time points, capture multi-scale information for each air-to-ground image, and obtain shallow and deep features of each air-to-ground image; select the air-to-ground image with the highest object segmentation quality based on the shallow and deep features;

[0062] The selected air-to-ground image is segmented to extract the landscape elements and their airspace features. Landscape elements include vegetation elements, water elements, and architectural elements. Airspace features include spatial location and spatial physical boundaries. The airspace features of each type of landscape element are aggregated and mapped in the same spatial dimension to generate multiple garden pattern distribution layers. Each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

[0063] Garden spaces can be, but are not limited to, parks, suburban forests, and other three-dimensional spaces that integrate plants, lakes, and artificial buildings. Garden spaces are characterized by large areas and three-dimensional interlacing. In order to improve the efficiency and comprehensiveness of garden space image acquisition, drones and other aerial photography equipment can be used to inspect and photograph garden spaces, obtaining air-to-ground images of the entire garden space. Figure 2 Consider an air-to-ground image of a municipal park. This image includes vegetation such as trees and lawns, water bodies such as artificial lakes and rivers, and artificial structures such as pavilions and steps. Vegetation, water bodies, and artificial structures each have distinct morphologies. Vegetation is in a growing state, its shape changing and continuously changing under external lighting and atmospheric flow. Water bodies are inherently fluid and also continuously changing under external lighting and atmospheric flow. Artificial structures are static relative to vegetation and water bodies, and their structural lines and outlines remain largely unchanged. The above analysis shows that the spatial distribution and morphology of landscape elements such as vegetation, water bodies, and artificial structures within a garden space vary significantly. Furthermore, different landscape elements change at varying speeds and degrees. A unified update of all landscape elements across the entire garden space would require not only a large amount of real-world data but also significant computing power and memory resources, making it impossible to quickly and accurately update the garden map. Therefore, independent updates can be performed for each of these three landscape elements: vegetation, water bodies, and artificial structures.

[0064] Considering that a single inspection and photography of a garden space may involve significant internal and external interference, in order to minimize the impact of interference, it is necessary to take aerial photos of the garden space at several time points to obtain multiple air-to-ground images. A multi-scale feature extraction network based on the Transformer architecture, such as the Adaptive Subspace Feature Fusion module, is used to capture multi-scale information from each air-to-ground image, obtaining both shallow and deep features of each air-to-ground image. Shallow features refer to spatial details such as the boundary outlines of objects within the image, while deep features refer to the specific semantic information of the image. Figure 3 , is to use the adaptive subspace feature fusion module to Figure 2 The shallow features (corresponding to Figure 3 (a)) and deep features (corresponding to Figure 3 (b)) to achieve multi-scale feature information representation for air-to-ground images. The total amount of feature information corresponding to each air-to-ground image is then determined based on the information content of all shallow and deep features within each air-to-ground image. The air-to-ground image with the highest total amount of feature information is selected as the one with the highest target segmentation quality, ensuring that the selected air-to-ground image contains sufficient detail information to meet target segmentation requirements.

[0065] The U-Net segmentation network can be used to segment the selected air-to-ground image, extracting the spatial positions and physical boundaries of the vegetation elements, water elements, and building elements in the air-to-ground image, thereby separately segmenting and extracting the three types of landscape elements in the air-to-ground image: vegetation, water, and buildings. The U-Net fusion network can also be used to aggregate and map all elements and their airspace characteristics under each type of landscape element in the same spatial dimension to generate multiple garden pattern distribution layers. That is, the U-Net fusion network is used to aggregate all vegetation elements and their airspace characteristics, all water elements and their airspace characteristics, and all building elements and their airspace characteristics in the same spatial coordinate dimension to generate three garden pattern distribution layers corresponding to vegetation elements, water elements, and building elements respectively. Please refer to Figure 4 (a), (b), and (c) are the garden pattern distribution layers corresponding to vegetation elements, water elements, and architectural elements, respectively. Each garden pattern distribution layer represents the spatial distribution density of vegetation elements, water elements, and architectural elements within the global grid range of the garden space. Among them, the darker the color of the grid in the garden pattern distribution layer, the greater the spatial distribution density of vegetation elements, water elements, and architectural elements.

[0066] Furthermore, in S100, the image disturbance feature of each garden pattern distribution layer is obtained to calibrate the expected update area of each garden pattern distribution layer, specifically:

[0067] Each garden pattern distribution layer is processed using a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer. Based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of each garden pattern distribution layer are determined to obtain the image disturbance characteristics.

[0068] The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

[0069] See also Figure 5 A cross-attention fusion network can be used to distribute redundant information, such as blank information and / or noise, across the entire image of each garden pattern distribution layer in both the horizontal and vertical dimensions. It is understood that the entire garden pattern distribution layer contains valid target information (such as the target's edge contour, chroma, brightness, and other information), blank information, and noise information. During the actual image update process, only this valid target information is considered, while the blank information and noise information do not contribute to the image update process and instead overwhelm the valid target information. The more blank information and noise information a grid subregion of the layer contains, the higher the degree of coverage of the valid target information in that grid subregion. Based on the distribution of redundant information in the horizontal and vertical dimensions of each garden pattern distribution layer, the coverage ratio of detailed information such as the target's edge contour, chroma, and brightness in each garden pattern distribution layer in both dimensions is determined and used as the image disturbance feature. Those skilled in the art can process the garden pattern distribution layer using the cross-attention fusion mechanism within the Python framework; this will not be described in detail here.

[0070] In addition, a threshold comparison is performed on the detail information coverage ratio value of each grid sub-area in each garden pattern distribution layer. If the detail information coverage ratio value exceeds the preset ratio threshold, the corresponding grid sub-area is determined as a grid sub-area with unclear details; otherwise, the corresponding grid sub-area is not determined as a grid sub-area with unclear details; and the area formed by connecting all the grid sub-areas with unclear details is calibrated as the expected update area of the garden pattern distribution layer, so as to limit the area range for subsequent use of real-scene images to update the layer content.

[0071] S200: Pre-processing the real-scene images of the garden space shared by the outside world to form multiple real-scene image libraries, and determining the mapping relationship between the desired update area and the real-scene image library; screening and integrating the real-scene images of the real-scene image library to obtain optimized real-scene images.

[0072] Furthermore, in S200, the real scene images of the garden space shared by the outside world are pre-processed to form multiple real scene libraries, and the mapping relationship between the desired update area and the real scene library is determined; the real scene images of the real scene library are screened and integrated to obtain the optimized real scene image, specifically:

[0073] Collect real-life images of garden spaces uploaded and shared by the outside world, identify and filter all collected real-life images, and retain some real-life images; the image features include image pixel parameter features and image element state features;

[0074] Perform spatial orientation recognition on each retained real-scene image, determine the spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the expected renewal area in the garden space, and determine the mapping relationship between the expected renewal area and the real-scene library;

[0075] According to the landscape element detail information of each real scene image in the real scene library, several real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and integrated from the screened real scene images, and the landscape elements are reorganized within the picture to obtain optimized real scene images.

[0076] Real-life images can be, but are not limited to, images captured by garden maintenance personnel using devices such as smartphones in the real environment of a garden space. Each real-life image corresponds to a real-life image of a local area within the garden space. By collecting all real-life images of the garden space uploaded and shared externally, each real-life image is subjected to image recognition to obtain pixel parameter characteristics such as image pixel resolution and image pixel contrast, as well as element status characteristics such as the number of landscape elements within the image and the degree of completeness of landscape elements (i.e., the ratio between the displayed portion of the landscape element in the image and the overall portion of the landscape element). Based on these image pixel parameter characteristics and image element status characteristics, all real-life images are screened, and real-life images with image pixel resolution less than a preset resolution threshold, image pixel contrast less than a preset contrast threshold, number of landscape elements within the image less than a preset number threshold, or landscape element completeness less than a preset degree threshold are eliminated, thereby retaining a portion of the real-life images.

[0077] Then, spatial orientation recognition is performed on each retained real-scene image, and the spatial orientation overlap angle value between any two retained real-scene images is determined, which is used as the spatial orientation association attribute between different retained real-scene images. If the spatial orientation overlap angle value between two retained real-scene images is greater than or equal to the preset angle threshold, the two retained real-scene images are divided into the same real-scene gallery; otherwise, the two retained real-scene images are divided into two different real-scene galleries. Through the above method, all retained real-scene images are distinguished to form multiple real-scene galleries, wherein all real-scene images under each real-scene gallery can be understood as being concentrated in the same spatial orientation corresponding to the garden space, and the overlapping angle interval of the spatial orientation range of all real-scene images under each real-scene gallery is determined as the main spatial orientation of each real-scene gallery. Through the above analysis, it can be seen that the main spatial orientation of the real-scene library represents the local area of the garden space that can be captured by all real-scene images in the real-scene library. To this end, the main spatial orientation of the real-scene library is compared with the distribution spatial orientation of the expected update area in the garden space. If the above main spatial orientation is consistent with the above distribution spatial orientation, the mapping relationship between the above real-scene library and the above expected update area is determined, which facilitates the subsequent accurate selection of a real-scene library from multiple real-scene libraries to provide landscape element data for filling and integrating the expected update area.

[0078] Furthermore, for all real-life images in each real-life image library, detailed information such as the edge contours of the landscape elements of each real-life image is obtained, and the detailed information such as the edge contours of the landscape elements of all real-life images is compared, and a number of real-life images in which no landscape element details overlap are screened. That is, for multiple real-life images in which the edge contours of the landscape elements overlap, only one real-life image is retained, so that the multiple retained real-life images do not overlap with each other in terms of details such as the edge contours of the landscape elements. The U-Net fusion network is then used to extract and fuse the three types of landscape elements, namely vegetation, water bodies, and buildings, from the real-life images that have been filtered and retained, and to reorganize them within the same picture space to obtain optimized real-life images. By using the above method, the landscape elements with the best quality in all real-life images in the same real-life image library are fused and reorganized to ensure that all landscape elements in the optimized real-life images are in the optimal visual viewing state.

[0079] S300: filling and fusing the optimized real scene image with the corresponding expected update area according to the mapping relationship; overlaying all garden pattern distribution layers that have completed the filling and fusion of the expected update area to obtain a garden map of the garden space.

[0080] In S300, based on the mapping relationship, the optimized real scene image is filled and fused with the corresponding desired update area; all garden pattern distribution layers that have completed the filling and fusion of the desired update area are superimposed to obtain a garden map of the garden space, specifically:

[0081] According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding expected update area;

[0082] According to the boundary position of the garden space, all the garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain the garden map of the garden space.

[0083] According to the above mapping relationship, the real scene library corresponding to the expected update area is determined, and the optimized real scene image corresponding to the above determined real scene library is extracted. The U-Net fusion network is then used to obtain the multi-scale spatial information of the background and landscape elements in the above optimized real scene image, wherein the multi-scale spatial information includes the size and contour shape information of the background and landscape elements in the horizontal and vertical dimensions. The U-Net fusion network is also used to refine the multi-scale spatial information to obtain embedded picture elements, so that the embedded picture elements are filled and fused into the corresponding expected update area; the embedded picture elements refer to vegetation, water bodies and / or architectural elements that are compatible with the above expected update area in chromaticity, brightness and contour texture. In addition, according to the boundary position of the garden space, all garden pattern distribution layers that have completed the filling and fusion of the expected update area are aligned and overlapped, and the three types of elements of vegetation, water bodies and buildings in the garden space are reintegrated into the same map space to obtain a garden map of the forest space, so as to achieve consistency between the map and the actual situation of the garden and improve the refinement of map information.

[0084] See also Figure 6 As shown, the present invention provides a garden map updating system based on real-scene image sharing, which includes the following modules:

[0085] A layer generation module is used to identify landscape elements in the air-to-ground image of the garden space, classify the landscape elements and associate them with space, and generate multiple garden pattern distribution layers;

[0086] An update region calibration module is used to obtain the image disturbance characteristics of each garden pattern distribution layer and to calibrate the expected update region of each garden pattern distribution layer;

[0087] The image library processing module is used to pre-process the real-scene images of the garden space shared by the outside world, form multiple real-scene image libraries, and determine the mapping relationship between the desired update area and the real-scene image library;

[0088] Image optimization module, used to filter and fuse real-scene images in the real-scene image library to obtain optimized real-scene images;

[0089] A filling and fusion module is used to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship;

[0090] The layer superposition module is used to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update area to obtain a garden map of the garden space.

[0091] Furthermore, the layer generation module is used to identify the landscape elements of the air-to-ground image of the garden space, classify the landscape elements and associate them with the space, and generate multiple garden pattern distribution layers, specifically:

[0092] Acquire air-to-ground images of the garden space at several time points, capture multi-scale information for each air-to-ground image, and obtain shallow and deep features of each air-to-ground image; select the air-to-ground image with the highest object segmentation quality based on the shallow and deep features;

[0093] Performing target segmentation on the selected air-to-ground image to extract the landscape elements and their spatial features; landscape elements include vegetation elements, water elements, and architectural elements; spatial features include spatial location and spatial physical boundaries; performing aggregation mapping of the spatial features of each type of landscape element in the same spatial dimension to generate multiple garden pattern distribution layers; each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element;

[0094] The update region calibration module is used to obtain the image disturbance features of each garden pattern distribution layer and to calibrate the expected update region of each garden pattern distribution layer. Specifically:

[0095] Each garden pattern distribution layer is processed using a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of each garden pattern distribution layer. Based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of each garden pattern distribution layer are determined to obtain the image disturbance characteristics.

[0096] The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

[0097] Furthermore, the image library processing module is used to pre-process the real-view images of the garden space shared by the outside world to form multiple real-view image libraries, and determine the mapping relationship between the desired update area and the real-view image library, specifically:

[0098] Collect real-life images of garden spaces uploaded and shared by the outside world, identify and filter all collected real-life images, and retain some real-life images; the image features include image pixel parameter features and image element state features;

[0099] Perform spatial orientation recognition on each retained real-scene image, determine the spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the expected renewal area in the garden space, and determine the mapping relationship between the expected renewal area and the real-scene library;

[0100] The image optimization module is used to filter and fuse the real-scene images in the real-scene library to obtain optimized real-scene images. Specifically:

[0101] According to the landscape element detail information of each real scene image in the real scene library, several real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and integrated from the screened real scene images, and the landscape elements are reorganized within the picture to obtain optimized real scene images.

[0102] Furthermore, the filling and fusion module is used to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship, specifically:

[0103] According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding expected update area;

[0104] The layer superposition module is used to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the desired update area to obtain the garden map of the garden space, specifically:

[0105] According to the boundary position of the garden space, all the garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain the garden map of the garden space.

[0106] The operation and effects of the garden map updating system based on real scene image sharing of the present invention are corresponding to and consistent with the above-mentioned garden map updating method based on real scene image sharing, and the garden map updating system based on real scene image sharing will not be repeated here.

[0107] In one embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0108] In one embodiment of the present invention, the present invention further provides a computer device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described above is implemented.

[0109] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding the necessary general-purpose hardware platform, or of course, by combining hardware and software. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A garden map updating method based on real-scene image sharing, characterized in that: The method comprises the following steps: S100: Identifying landscape elements of an air-to-ground image of a garden space, classifying and spatially associating the landscape elements, and generating multiple garden pattern distribution layers; obtaining image disturbance features of each garden pattern distribution layer, and using them to calibrate a desired update area of each garden pattern distribution layer; S200: Pre-processing the real-scene images of the garden space shared by the outside world to form multiple real-scene image libraries, and determining the mapping relationship between the desired update area and the real-scene image library; screening and fusing the real-scene images of the real-scene image library to obtain an optimized real-scene image; S300: filling and fusing the optimized real scene image with the corresponding expected update area according to the mapping relationship; and superimposing all garden pattern distribution layers in which the expected update area is filled and fused to obtain a garden map of the garden space.

2. The method according to claim 1, characterized in that In S100, the landscape elements of the air-to-ground image of the garden space are identified, the landscape elements are classified and spatially associated, and multiple garden pattern distribution layers are generated, specifically: Obtain air-to-ground images of the garden space at several time points, capture multi-scale information for each air-to-ground image, and obtain shallow and deep features of each air-to-ground image; selecting an air-to-ground image with the highest object segmentation quality according to the shallow features and the deep features; The selected air-to-ground image is segmented to extract the landscape elements and their spatial features; wherein the landscape elements include vegetation elements, water elements, and architectural elements; the spatial features include spatial position and spatial physical boundaries; the spatial features of each type of landscape element are aggregated and mapped in the same spatial dimension to generate multiple garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element.

3. The method according to claim 1, characterized in that In S100, the image disturbance features of each garden pattern distribution layer are obtained to calibrate the expected update area of each garden pattern distribution layer, specifically: Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of the image of each garden pattern distribution layer; based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of the image of each garden pattern distribution layer are determined to obtain the image disturbance characteristics; The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

4. The method according to claim 1, wherein In S200, the real-scene images of the garden space shared by the outside world are pre-processed to form multiple real-scene libraries, and the mapping relationship between the desired update area and the real-scene library is determined; the real-scene images of the real-scene library are filtered and integrated to obtain the optimized real-scene images, specifically: Collecting real-life images of garden spaces uploaded and shared by the outside world, identifying and screening all collected real-life images, and retaining some real-life images; wherein the image features include image pixel parameter features and image element state features; Perform spatial orientation recognition on each retained real-scene image, determine spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the desired update area in the garden space, and determine a mapping relationship between the desired update area and the real-scene library; According to the landscape element detail information of each real scene image in the real scene library, a number of real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and merged from the screened real scene images, and the landscape elements are reorganized within the picture to obtain an optimized real scene image.

5. The method according to claim 1, wherein In S300, based on the mapping relationship, the optimized real scene image is filled and fused with the corresponding desired update area; all garden pattern distribution layers that have completed the filling and fusion of the desired update area are superimposed to obtain a garden map of the garden space, specifically: According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding desired update area; According to the boundary position of the garden space, all garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain a garden map of the garden space.

6. The garden map updating system based on real-scene image sharing is characterized by: The system includes the following modules: A layer generation module is used to identify landscape elements in the air-to-ground image of the garden space, classify the landscape elements and associate them with space, and generate multiple garden pattern distribution layers; An update region calibration module is used to obtain the image disturbance characteristics of each garden pattern distribution layer and to calibrate the expected update region of each garden pattern distribution layer; A picture library processing module is used to pre-process the real-scene images of the garden space shared by the outside world to form multiple real-scene picture libraries, and determine the mapping relationship between the desired update area and the real-scene picture library; An image optimization module, configured to filter and fuse the real-scene images in the real-scene image library to obtain optimized real-scene images; A filling and fusion module, configured to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship; The layer superposition module is used to superimpose all the garden pattern distribution layers that have completed the filling and fusion of the expected update area to obtain a garden map of the garden space.

7. The system according to claim 6, characterized in that The layer generation module is used to identify the landscape elements of the air-to-ground image of the garden space, classify the landscape elements and associate them with the space, and generate multiple garden pattern distribution layers, specifically: Obtain air-to-ground images of the garden space at several time points, capture multi-scale information for each air-to-ground image, and obtain shallow and deep features of each air-to-ground image; selecting an air-to-ground image with the highest object segmentation quality according to the shallow features and the deep features; Performing target segmentation on the selected air-to-ground image to extract landscape elements and their spatial features; wherein the landscape elements include vegetation elements, water elements, and architectural elements; and the spatial features include spatial location and spatial physical boundaries; performing aggregation mapping of the spatial features of each type of landscape element in the same spatial dimension to generate multiple garden pattern distribution layers; wherein each garden pattern distribution layer corresponds to the spatial layout of only one type of landscape element; The update region calibration module is used to obtain the image disturbance features of each garden pattern distribution layer, and to calibrate the expected update region of each garden pattern distribution layer, specifically: Each garden pattern distribution layer is processed by a cross-attention fusion mechanism to obtain the redundant information distribution of the horizontal and vertical dimensions of the image of each garden pattern distribution layer; based on the redundant information distribution, the detail coverage ratio characteristics of the horizontal and vertical dimensions of the image of each garden pattern distribution layer are determined to obtain the image disturbance characteristics; The detail coverage ratio characteristics of each garden pattern distribution layer are compared with the threshold of each grid sub-region of the picture one by one to calibrate the expected update area of each garden pattern distribution layer.

8. The system according to claim 6, wherein: The image library processing module is used to pre-process the real-view images of the garden space shared by the outside world to form multiple real-view image libraries, and determine the mapping relationship between the expected update area and the real-view image library, specifically: Collecting real-life images of garden spaces uploaded and shared by the outside world, identifying and screening all collected real-life images, and retaining some real-life images; wherein the image features include image pixel parameter features and image element state features; Perform spatial orientation recognition on each retained real-scene image, determine spatial orientation correlation attributes between different retained real-scene images, and thereby distinguish all retained real-scene images to form multiple real-scene libraries; compare the main spatial orientation of each real-scene library with the distribution spatial orientation of the desired update area in the garden space, and determine a mapping relationship between the desired update area and the real-scene library; The image optimization module is used to filter and fuse the real-scene images in the real-scene image library to obtain optimized real-scene images, specifically: According to the landscape element detail information of each real scene image in the real scene library, a number of real scene images without overlapping landscape element details are screened; multiple types of landscape elements are extracted and merged from the screened real scene images, and the landscape elements are reorganized within the picture to obtain an optimized real scene image.

9. The system according to claim 6, wherein: The filling and fusion module is used to fill and fuse the optimized real scene image with the corresponding expected update area according to the mapping relationship, specifically: According to the mapping relationship, an optimized real scene image of a matching real scene library is selected; multi-scale spatial information of background and landscape elements in the selected optimized real scene image is obtained, and the multi-scale spatial information is refined to obtain embedded picture elements, thereby filling and fusing the embedded picture elements into the corresponding desired update area; The layer superposition module is used to superimpose all garden pattern distribution layers that have completed the filling and fusion of the desired update area to obtain a garden map of the garden space, specifically: According to the boundary position of the garden space, all garden pattern distribution layers that have completed the filling and fusion of the desired update area are aligned and superimposed to obtain a garden map of the garden space.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.

Images