Urban streetscape information retrieval method

Abstract

A city street view information retrieval method comprises the following steps: S100, street view data acquisition: acquiring whole city street view image information and corresponding geographic latitude and longitude information through a Google map API (Application Program Interface); s200, visual features of the streetscape images are extracted and quantified, multi-dimensional extraction and quantitative analysis are carried out on the visual features of the streetscape images, and a comprehensive visual representation system is constructed and comprises visual entity representation, visual impression representation and vision field structure representation; s300, comprehensive training of streetscape data and formation of an intelligent search system: adopting a self-organizing mapping neural network algorithm to carry out comprehensive training on the streetscape visual representation system data set, constructing a structured and visual streetscape feature distribution diagram, and forming a core system of the intelligent search system; and S400, using an intelligent search system, including streetscape visual information retrieval, visual display and interactive exploration. And visual and efficient urban street view data query and analysis services are provided.

Description

Technical Field

[0001] The present invention relates to the field of geographic information, and in particular to a method for retrieving urban street view information. Background Art

[0002] Street View technology is a crucial extension of geographic information services. By combining real-world street imagery with geographic information technology, it provides users with a virtualized street view browsing experience. Currently, Street View technology has become a fundamental tool for urban navigation, virtual tourism, and urban landscape display. Its key technical processes include the collection of street view data, its integration with geographic information, and the interactive display of street view imagery.

[0003] With the accelerated development of smart cities, the demand for urban street view information has far surpassed simple virtual tours and visualization experiences. While existing street view applications can provide basic street visualization services, they still face significant technical bottlenecks and shortcomings in the following areas: 1. Limited utilization of street view data. Existing street view applications primarily focus on the collection and static display of street view data, lacking in-depth analysis and intelligent utilization of street view data, and thus unable to meet the complex spatial analysis needs of smart cities. 2. Incomplete visual feature extraction. Existing technologies often rely on single features, making it difficult to systematically and comprehensively quantify the multi-dimensional visual characteristics of street views, resulting in a one-sided representation of street view information.

[0004] Based on these deficiencies in technology and application, there is an urgent need for a tool that can efficiently integrate, analyze and display urban street view information to meet the multi-level needs of everyone from ordinary users to urban planners. Summary of the Invention

[0005] In response to the above-mentioned defects, the purpose of the present invention is to propose a street view information retrieval method, which, by combining image recognition and geographic information technology, provides users with personalized and customized street view retrieval and display services, thereby achieving comprehensive mining and efficient utilization of street view information.

[0006] To achieve this object, the present invention adopts the following technical solutions:

[0007] A method for retrieving urban street view information comprises the following steps:

[0008] S100, street view data acquisition, obtaining a city's entire street view image information and its corresponding geographic longitude and latitude information through the Google Maps API interface;

[0009] S200, Street View Image Visual Feature Extraction and Quantification, performs multi-dimensional extraction and quantitative analysis of the visual features of street view images to construct a comprehensive visual representation system, including three categories: visual entity representation, visual impression representation, and visual field structure representation;

[0010] S300: Comprehensive training of street view data and formation of an intelligent search system. This involves using a self-organizing map neural network algorithm to comprehensively train the street view visual representation system dataset, constructing a structured and visual street view feature distribution map, and forming the core system of the intelligent search system.

[0011] S400, the use of intelligent search system, establishes a search system with visual feature data set as the core, and provides intuitive and efficient urban street view data query and analysis services through street view visual information retrieval and urban street view panorama display and interactive exploration.

[0012] Preferably, in step S200, the three categories of visual representations are subdivided into 19-dimensional visual feature vectors according to the characteristics of the street view image, specifically including:

[0013] Visual entity representation:

[0014] Building structure, used to detect the visual proportion of artificial buildings in street view images;

[0015] Street Furniture, which detects the visual proportion of street furniture infrastructure in street view imagery;

[0016] Openness, used to detect the visual proportion of sky in street view images;

[0017] Green view rate, used to detect the visual ratio of vegetation in street view images;

[0018] Walkable roads, used to detect the visual ratio of walkable roads in street view images;

[0019] Roadway, used to detect the visual ratio of walkable roads in street view images;

[0020] Multi-purpose sites, which detects the visual proportion of sites that can be used for multi-purpose purposes in street view imagery;

[0021] Human activity, used to detect the visual proportion of people in street view images;

[0022] Traffic activity, which detects the visual proportion of vehicles in street view images;

[0023] Landscape, used to detect the visual proportion of landscape in street view images;

[0024] Visual impression representation:

[0025] Texture complexity, which is used to detect the edge complexity of all details in street view images;

[0026] Layout complexity, which is used to detect the number of elements and information uncertainty of the basic composition of street view images;

[0027] Color complexity, used to detect the richness of color composition in street view images;

[0028] Visual field structure representation:

[0029] Viewshed area is used to measure the visible space area from a certain viewpoint on the street and is related to the connectivity of the space;

[0030] Viewshed perimeter, used to measure the visible viewshed boundary perimeter of a certain viewpoint on the street;

[0031] Sightline compactness measures the compactness of the visual field at a certain viewpoint on the street, reflecting the shape attributes of the visible space and the continuity and consistency of the experience. The closer the visual field space is to a circle, the higher the visual field compactness.

[0032] View occlusion measures the length of the edge of the view space that is not defined by entities at a certain viewpoint on the street, quantifies the degree of influence of obstructions on the extension of vision, and reflects the visual transparency of the space.

[0033] Viewshed skewness measures the degree of difference in radial sight length at a certain viewpoint on the street and quantifies the symmetry and uniformity of the viewshed space;

[0034] View drift measures the distance and direction from a certain viewpoint on the street to the center point of its view space.

[0035] Preferably, step S300 specifically includes the following steps:

[0036] S310, feature standardization of street view visual data. To improve the convergence of the model and the stability of the search results, the z-score algorithm is first applied to standardize all input parameters to ensure that each feature has a consistent weight within the same numerical range.

[0037] The z-score formula is:

[0038]

[0039] Among them, x i is the i-th eigenvalue of the original data, μ is the data mean, and σ is the data standard deviation;

[0040] S320, self-organizing map neural network algorithm is used for model training. By training the input data with the self-organizing map neural network algorithm, the model can learn the intrinsic structure of the data, map high-dimensional data into a two-dimensional feature space, and retain the topological structure of the similarity relationship between the data.

[0041] Preferably, step S320 is specifically as follows:

[0042] S321, initial network construction, setting the network node structure (30*30) of the self-organizing map neural network algorithm, and randomly initializing the weight vector of each node;

[0043] S322, weight update. During the training process, the Euclidean distance between the input data and the network node is calculated to map the input data to the network node closest to it, and the weight vector of the node and its neighborhood is updated. The formula is:

[0044] I BMU =argmin{||sv k (t)||}

[0045] v k (t+1)=v k (t)+α(t)h bk [sv k (t)]

[0046] Among them, I BMU It is the network node closest to the original data generated in each update process, v k (t) represents the updated weight at time t, s represents the original data, h bk is the Gaussian neighborhood function;

[0047] S323, cluster formation, after multiple iterative training, similar street view data are clustered to adjacent network nodes to form a meaningful cluster distribution.

[0048] Furthermore, based on the search system, the "addressing" search function can be expanded. The addressing search function is mainly used to quickly locate the geographical location of the street view in the city based on the user-defined street view visual features; users can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

[0049] Furthermore, based on the search system, a "scene search" function can be expanded. The scene search function helps users understand the street view types and distribution characteristics in the target area based on the geographical location; users select the area of ​​interest on the geographic map to view the street view types, quantity and characteristic distribution contained in the area, thereby supporting rapid street view auditing and analysis.

[0050] Furthermore, the search system can be expanded to include a "landscape relationship" search function. This function is used to establish a visual association between the overall street view image and the geographic map, helping users to intuitively understand the spatial distribution of different types of street views in the city. Through color coding, the search system will simultaneously display street view types on the two maps, allowing users to gain a macro-level understanding of the relationship between street view characteristics and geographic locations.

[0051] In the search system interface, street view types in the overall street view map will be assigned different color tags to distinguish different types of street views. Corresponding color tags will appear on the geographic map, clearly showing the distribution of these street view types in the city.

[0052] Furthermore, the search system can be expanded to include a "Scenic Spot Preference" search function. This personalized street view selection tool is based on the user's personal interests. Users can find a street view type that matches their interests within the overall street view map and mark it as a preferred center point. This determines the user's street view interest point, and using this preference information as the center reference of the heat map, the system automatically generates a color spectrum marker and displays the entire street view map in a heat distribution based on the degree of similarity. The color spectrum gradually changes from warm to cool, indicating the degree of street view preference match: warm tones represent street views that highly match the user's preferences, while cool tones represent street views with a lower degree of match.

[0053] Furthermore, in the “visual entity features”, building structure, street furniture, openness, green view rate, walkable roads, driveways, multi-functional venues, human activities, traffic vitality, and landscape are all calculated using the pixel ratio, that is, the proportion of the feature in the total pixels of the entire image.

[0054] Furthermore, in the “visual impression features”, texture complexity is measured using the fractal dimension box-counting algorithm;

[0055] The layout complexity is measured using the image two-dimensional entropy algorithm;

[0056] Color complexity was measured using the color complexity measurement;

[0057] Furthermore, in the “viewshed structure characteristics”, the viewshed area vector is measured using the viewshed area calculation method;

[0058] The viewshed perimeter vector is measured using the viewshed perimeter calculation method;

[0059] The sightline compactness vector is calculated using the viewshed compactness calculation method;

[0060] The view occlusion vector is calculated using the view occlusion calculation method;

[0061] The viewshed skewness vector is calculated using the viewshed line of sight skewness calculation method;

[0062] The viewshed drift vector is calculated using the viewshed drift calculation method.

[0063] One of the above-mentioned technical solutions has the following beneficial effects: through the above-mentioned steps, the solution utilizes machine learning and big data analysis technologies to achieve comprehensive and rapid data collection of urban street scenes; by integrating features from multiple dimensions, such as visual entity representation, visual impression representation, and visual field structure representation, it provides a more comprehensive and multi-level visual semantic understanding of urban street scenes; by utilizing self-organizing map neural network technology, it effectively identifies the inherent connections and potential patterns in street scene big data, improving the intelligence of street scene retrieval; by constructing a search platform, it provides an intuitive visual interface and customized interactive query functions, enabling users to efficiently and conveniently obtain the required street scene information. Overall, the solution achieves in-depth mining and accurate retrieval of multi-dimensional, large-scale urban street scene data, effectively improving the efficiency and scientific nature of urban governance, planning, and management, and has significant innovative value and wide applicability in multiple application scenarios such as intelligent transportation, tourism planning, business analysis, and urban sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the overall process;

[0065] Figure 2 It is a flowchart of the self-organizing map neural network algorithm for model training;

[0066] Figure 3 It is a specific process diagram formed by the intelligent search system;

[0067] Figure 4 It is a schematic diagram of the corresponding index relationship between visual parameters, street view image, street view function and geographic information function.

[0068] Figure 5 It is the operating interface of the intelligent search system;

[0069] Figure 6-Figure 7 It uses the street view search and addressing based on visual parameters in the addressing search function;

[0070] Figure 8-Figure 9 It is the addressing of custom street view;

[0071] Figure 10-11 It is street view addressing based on POI function;

[0072] Figure 12-13 It is the viewfinder function;

[0073] Figure 14 It is a landscape relationship;

[0074] Figure 15-16 It is the preference of scenery;

[0075] Figure 17 This is an overview of the "Visual Entity Representation" search parameters for the Street View search system;

[0076] Figure 18 It is an overview of the search parameters of the "Visual Impression Representation" of the Street View search system;

[0077] Figure 19 It is an overview of the "View Structure Representation" search parameters of the Street View search system;

[0078] Figure 20 It is an overview of the search parameters of the "Street View POI Function" of the Street View search system; DETAILED DESCRIPTION

[0079] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0080] like Figure 1 As shown, a method for retrieving urban street view information includes the following steps:

[0081] S100, street view data acquisition, obtaining a city's entire street view image information and its corresponding geographic longitude and latitude information through the Google Maps API interface;

[0082] S200, Street View Image Visual Feature Extraction and Quantification, performs multi-dimensional extraction and quantitative analysis of the visual features of street view images to construct a comprehensive visual representation system, including three categories: visual entity representation, visual impression representation, and visual field structure representation;

[0083] S300: Comprehensive training of street view data and formation of an intelligent search system. This involves using a self-organizing map neural network algorithm to comprehensively train the street view visual representation system dataset, constructing a structured and visual street view feature distribution map, and forming the core system of the intelligent search system.

[0084] S400, the use of intelligent search system, establishes a search system with visual feature data set as the core, and provides intuitive and efficient urban street view data query and analysis services through street view visual information retrieval and urban street view panorama display and interactive exploration.

[0085] Through the above steps, the solution uses machine learning and big data analysis technologies to achieve comprehensive and rapid data collection of urban street scenes; by integrating multiple dimensions of features such as visual impression representation, visual entity representation, and field of view structure representation, it provides a more comprehensive and multi-level visual semantic understanding of urban street scenes; using self-organizing map neural network technology, it effectively identifies the inherent connections and potential patterns of street scene big data, improving the intelligence of street scene retrieval; by constructing a search platform, it provides an intuitive visual interface and customized interactive query functions, enabling users to obtain the required street scene information efficiently and conveniently.

[0086] Overall, the solution achieves in-depth mining and accurate retrieval of multi-dimensional, large-scale urban street scene data, effectively improving the efficiency and scientific nature of urban governance, planning, and management. It has significant innovative value and wide applicability in multiple application scenarios such as intelligent transportation, tourism planning, business analysis, and urban sustainable development.

[0087] The system uses an efficient street view data acquisition method to extract urban street view images and related geographic information from the entire city by calling the API interface of existing map data. The street view acquisition parameters are as follows:

[0088] Collection accuracy: Street view image data is collected at 50-meter intervals to ensure the continuity and integrity of street coverage.

[0089] Image size: The resolution of the captured image is set to 512 × 512 pixels to meet the requirements of efficient storage and computing while taking into account image clarity.

[0090] Viewing angle settings:

[0091] Horizontal heading: Set to 0°, which means the street view is facing the road ahead.

[0092] Vertical viewing angle (pitch): Set to 20°, tilting slightly upward to obtain an overall view of the ground and buildings.

[0093] Field of view (FOV): Set to 120° to ensure a larger horizontal field of view coverage and capture more street view information.

[0094] Based on the above parameters, street view data of 81,478 cities are obtained.

[0095] In step S200, the three categories of visual representations are subdivided into 19-dimensional visual feature vectors based on the characteristics of the street view image, specifically including:

[0096] Visual entity representation:

[0097] Building structure, used to detect the visual proportion of artificial buildings in street view images;

[0098] Street Furniture, which detects the visual proportion of street furniture infrastructure in street view imagery;

[0099] Openness, used to detect the visual proportion of sky in street view images;

[0100] Green view rate, used to detect the visual ratio of vegetation in street view images;

[0101] Walkable roads, used to detect the visual ratio of walkable roads in street view images;

[0102] Roadway, used to detect the visual ratio of walkable roads in street view images;

[0103] Multi-purpose sites, which detects the visual proportion of sites that can be used for multi-purpose purposes in street view imagery;

[0104] Human activity, used to detect the visual proportion of people in street view images;

[0105] Traffic activity, which detects the visual proportion of vehicles in street view images;

[0106] Landscape, used to detect the visual proportion of landscape in street view images;

[0107] Visual impression representation:

[0108] Texture complexity vector, used to detect the edge complexity of all details in street view images;

[0109] Layout complexity vector, used to detect the number of elements and information uncertainty of the basic composition of street view images;

[0110] Color complexity vector, used to detect the richness of color composition in street view images;

[0111] Visual field structure representation:

[0112] The view area vector is used to measure the visible space area of ​​a certain viewpoint on the street and is related to the connectivity of the space;

[0113] View perimeter vector, used to measure the visible view boundary perimeter of a certain viewpoint on the street;

[0114] The sightline compactness vector measures the compactness of the view from a certain viewpoint on the street, reflecting the shape attributes of the visible space and the continuity and consistency of the experience. The closer the viewspace is to a circle, the higher the sightline compactness.

[0115] The view occlusion vector measures the length of the edge not defined by entities in the view space of a certain viewpoint on the street, quantifies the degree of influence of obstructions on the extension of vision, and reflects the visual transparency of the space.

[0116] The viewshed skewness vector measures the difference in radial sight length at a certain viewpoint on the street and quantifies the symmetry and uniformity of the viewshed space.

[0117] The view drift vector measures the distance and direction from a certain viewpoint on the street to the center point of its view space.

[0118] Based on the above digital representation framework for street view visual information, street view search parameters were formed. These parameters control the weight and magnitude of each street view visual feature by adjusting the value, providing a precise data foundation for intelligent retrieval and personalized services. Based on the above 19-dimensional comprehensive digital representation framework for street view visual information, a more comprehensive multidimensional parameter system for urban street views was innovatively constructed. The multidimensional features not only meticulously cover multiple dimensions of visual entities, visual impressions, and visual field structure, but also enable flexible control of the weight of each visual feature type by precisely adjusting the value of each feature. This framework can capture and distinguish richer and more complex street view details, enhancing the accuracy and flexibility of information retrieval and achieving a more accurate and personalized query experience. In addition, this detailed feature decomposition lays a solid data foundation for subsequent intelligent analysis and applications, such as urban planning optimization and environmental assessment.

[0119] like Figure 2 As shown, in step S300, the following steps are specifically included:

[0120] S310, feature standardization of street view visual data. To improve the convergence of the model and the stability of the search results, the z-score algorithm is first applied to standardize all input parameters to ensure that each feature has a consistent weight within the same numerical range.

[0121] The z-score formula is:

[0122]

[0123] Among them, x i is the i-th eigenvalue of the original data, μ is the data mean, and σ is the data standard deviation;

[0124] S320, self-organizing map neural network algorithm is used for model training. By training the input data with the self-organizing map neural network algorithm, the model can learn the intrinsic structure of the data, map high-dimensional data into a two-dimensional feature space, and retain the topological structure of the similarity relationship between the data.

[0125] This topological feature distribution map can effectively identify the intrinsic associations and grouping characteristics of different street view data, reflect the visual distribution patterns of street view features, facilitate the efficient operation of subsequent retrieval and analysis modules, and provide more structured and hierarchical feature support for intelligent search systems.

[0126] like Figure 3 As shown, step S320 is specifically as follows:

[0127] S321, initial network construction, setting the network node structure (30*30) of the self-organizing map neural network algorithm, and randomly initializing the weight vector of each node;

[0128] S322, weight update. During the training process, the Euclidean distance between the input data and the network node is calculated to map the input data to the network node closest to it, and the weight vector of the node and its neighborhood is updated. The formula is:

[0129] I BMU =argmin{||sv k (t)||}

[0130] v k (t+1)=v k (t)+α(t)h bk [sv k (t)]

[0131] Among them, I BMU It is the network node closest to the original data generated in each update process, v k (t) represents the updated weight at time t, s represents the original data, h bk is the Gaussian neighborhood function;

[0132] S323, cluster formation, after multiple iterative training, similar street view data are clustered to adjacent network nodes to form a meaningful cluster distribution.

[0133] like Figure 4As shown, through the aforementioned data training process, the Self-Organizing Map neural network algorithm links 19-dimensional visual parameters with their corresponding streetscape images. Combining the Self-Organizing Map neural network data with streetscape geographic information, it establishes an index relationship between "visual parameters" - "streetscape image" - "streetscape function" - "geographic information." Furthermore, the comprehensive streetscape image generated by the Self-Organizing Map neural network algorithm reveals the essential logic and connections of streetscape visuals, establishing clear structured connections between streetscape visual types, clustering relationships, and neighborhood characteristics, providing data-driven support for exploring potential streetscape patterns. This results in a streetscape search system. In summary, the advantages of the Self-Organizing Map neural network algorithm lie in its powerful unsupervised learning capabilities and topology-preserving properties, enabling the effective representation and organization of high-dimensional data in a low-dimensional space. This allows complex and fragmented streetscape information to be integrated into a feature distribution with clear logical relationships and intuitive structure. This feature distribution not only intuitively expresses the similarities and differences between streetscape visual features, but also reveals the inherent patterns and logic of streetscape images through clustering and visualization, providing a new structured perspective for understanding and analyzing streetscape information. This effectively supports further analysis of streetscape evolution and distribution patterns, demonstrating unique advantages in exploring urban spatial design, environmental optimization, and functional zoning. Crucially, by integrating streetscape visual information, a multi-level index relationship is constructed, connecting "visual parameters" - "streetscape image" - "streetscape function" - and "geographic information." This not only provides clear typological classification and reveals neighborhood characteristics for streetscape data, but also enables the system to accurately query and dynamically evaluate streetscape data from multiple angles and levels during subsequent retrieval and analysis. This indexing system not only provides data-driven support for the discovery of streetscape patterns and the exploration of potential relationships, but also lays a solid technical foundation for the subsequent construction of an intelligent and personalized streetscape search platform. Through this series of data preprocessing and clustering processes, this step ultimately achieves significant improvements in streetscape data type identification, grouping, and visualization, effectively ensuring the efficiency and accuracy of urban streetscape information retrieval.

[0134] like Figure 5 Shown is the interface of the street view search system.

[0135] like Figure 6-Figure 7 As shown, the "addressing" function in the street view search system is mainly used to quickly locate the geographical location of the street view in the city based on the user-defined street view visual features; the user can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

[0136] In the left panel, users can select and adjust various visual feature parameters, such as building structure, street furniture, and green view ratio. These parameters provide a search basis for addressing, allowing users to freely combine features to precisely define the streetscape type they are interested in. For example, users can set high green view ratio and high openness to locate areas with dense natural landscapes, or low green view ratio and high building density to find urban areas. Based on the user-defined visual feature parameters, the system will highlight streetscapes that meet the criteria in the "Street View Overview Map." Clicking the "Address" button will display the actual location of these streetscapes in the city on the "Geographic Map." This feature is suitable for multiple application scenarios, such as: Urban planning and design: Urban planners can use the addressing function to filter areas with specific streetscape characteristics based on specified visual features. Real estate and commercial site selection analysis: Real estate developers and businesses can use the addressing function to select locations based on specified streetscape feature parameters. Retailers can select streetscape areas with high pedestrian traffic as potential location references to increase store visibility and foot traffic.

[0137] like Figure 8-Figure 9 As shown, the addressing of custom street view:

[0138] Users can manually select certain specific visual feature combinations in the "Street View Panorama". After clicking the "Address" button, the system will mark the distribution locations of street views that meet the conditions on the "Geographic Map". This function supports a variety of personalized needs, such as: Outdoor activity preference filtering: Outdoor sports enthusiasts can choose open public space street views, such as squares or parks. The system will display the urban distribution of these places to help users find areas suitable for exercise. Tourist point of interest search: Tourists can choose street views with high green view rates, moderate building structures, and high street furniture features, which makes it easier to find areas with rich natural landscapes and suitable for rest in the city, providing tourists with a more pleasant urban experience.

[0139] like Figure 10-11 As shown, street view addressing based on POI function:

[0140] Street view addressing based on the POI function on the left side of the interface can help users accurately locate urban areas that meet specific needs based on the dual criteria of visual features and functional attributes. Users can first filter out street views with specific visual features by adjusting visual parameters, or directly select a specific type of street view in the overall street view. The system will mark the distribution of these street views in the city on the geographic map based on the filtering conditions. Afterwards, users can further select specific functional attributes (such as catering services, shopping services, financial services, etc.) as secondary filtering conditions. The system will automatically filter out street views that do not meet the functional attributes, and finally present the distribution of street views that meet both visual features and specific functions. This function not only provides users with a multi-level screening path, but also significantly improves the accuracy and depth of urban space exploration. Whether it is used for urban planning, functional area analysis, or finding street view places with specific functions in a specific area, users can use this combined screening mechanism to obtain intuitive and detailed results to meet complex urban research and application needs.

[0141] The operating steps of the addressing search function are as follows: 1. Enter the search system: users select the Chinese or English interface through the platform's start page, enter the overview page, understand the platform functions, and then enter the main operation page; 2. Visual parameter setting: users select "Visual Impression Representation" in the visual parameter adjuster, check and adjust the "Texture Density", "Layout Complexity" and "Color Richness" parameters, and set them to "High", "Medium" and "High" respectively; 3. Street view search: According to the parameter settings, the platform immediately highlights the street scenes that meet the above visual characteristics in the overall street view. Users click the "Addressing" button to locate the actual location of these street scenes on the geographic map; 4. Result analysis: users can see that street scenes with high texture density, high color richness and medium layout complexity are mainly concentrated in the city's commercial and tourist areas.

[0142] like Figure 12-13 As shown, based on the search system, a "scene search" function can be expanded to help users understand the street view types and distribution characteristics in the target area based on the geographical location; users can select the area of ​​interest to view the street view types, quantity and characteristic distribution contained in the area, thereby supporting rapid street view auditing and analysis.

[0143] The operating steps of the scene search function are as follows: 1. Area selection: Users use the polyline tool on the geographic map to select an area of ​​the city, such as the area around the city's Central Park; 2. Heat map generation: After clicking the "Scene Search" button, the platform generates a heat map in the street view map to show the composition types and quantity of street scenes in the selected area; 3. Result analysis: The heat map shows that the area around Central Park has a high concentration of street scenes with high green view rates and high openness, indicating that the area has a high green coverage rate and is suitable for leisure and outdoor activities.

[0144] like Figure 14 As shown, based on the search system, a "landscape relationship" search function can be expanded. The landscape relationship search function is used to establish a visual association between the street view map and the geographic map, helping users to intuitively understand the spatial distribution of different types of street scenes in the city; through color marking, the search system will simultaneously display the street view type on the two maps, allowing users to understand the relationship between street view characteristics and geographic location from a macro perspective;

[0145] In the search system interface, street view types in the overall street view map will be assigned different color tags to distinguish different types of street views. Corresponding color tags will appear on the geographic map, clearly showing the distribution of these street view types in the city.

[0146] By displaying color markers simultaneously, users can quickly determine the primary distribution areas of a particular streetscape type within a city. For example, a green marker may represent an area with abundant greenery, while a yellow marker represents an area with high building density. This feature is well-suited for scenarios such as: Streetscape distribution feature analysis: helping urban planners analyze the layout of streetscapes with different visual characteristics within the city, providing a reference for block design and regional planning. The "Landscape Relationship" feature establishes a visual connection between the overall streetscape image and the geographic map, allowing users to explore streetscape distribution features within a broader urban context, providing an important platform support for urban design, landscape analysis, and spatial planning.

[0147] The operation steps of the landscape relationship function are as follows: 1. Color tag setting: The user clicks the "Landscape Relationship" button, and the platform assigns different color tags to different street view types on the street view overview image and geographic map; 2. Spatial distribution analysis: The user can quickly identify areas with abundant greenery (green tags) and high-density building areas (gray tags) through the simultaneous display of color tags; 3. Result analysis: The user can intuitively see the spatial distribution of green areas and densely built areas in the city, providing a reference for urban planning and landscape design.

[0148] like Figure 15-16 As shown, the search system can be expanded to include a "Scenic Spot Preference" search function. This personalized street view selection tool is based on the user's personal interests. Users find street view types that match their interests in the search system and mark them as their preferred center points. This determines the user's street view points of interest, and using this preference information as the center reference of the heat map, the system automatically generates color spectrum markers and displays the entire street view map in a heat distribution based on the degree of similarity. The color spectrum gradually changes from warm to cool, indicating the degree of street view preference match: warm tones represent street views that highly match the user's preferences, while cool tones represent street views with a lower degree of match.

[0149] Through color spectrum visualization, users can quickly identify urban areas that match their preferences. In the "Geographic Map," the system displays street view areas that meet the user's preferences as colored dots on the city map. This preference mapping allows users to more intuitively locate and identify distribution areas that meet their preferences in geographic space, making it easier to find suitable street view environments during actual urban exploration and planning.

[0150] This feature helps users quickly identify areas that meet their personal preferences within a wide range of street view distribution, providing efficient support for urban exploration, analysis, and personalization needs.

[0151] The operation steps of the scene preference function are as follows: 1. Preference setting: users find street view types that match their personal interests in the street view map, such as park areas with high green view rate and high openness, and mark them as preference center points; 2. Thermal spectrum generation: After clicking the "Scenic Preference" button, the platform generates a thermal spectrum based on the user's preference settings, showing the street view areas that match the user's preferences; 3. Result analysis: users can see that street view areas that highly match their personal preferences are displayed in warm colors (yellow) in the street view map, and highlighted as color points on the geographic map, making it easy for users to quickly identify and access them.

[0152] In addition, the 19-dimensional visual features include:

[0153] “Visual entity features”: building structure, street furniture, openness, green view rate, walkable roads, roadways, multi-functional venues, human activities, traffic vitality, and landscape are all calculated using pixel ratio, that is, the proportion of the feature in the total pixels of the entire image.

[0154] “Visual impression representation”: Texture complexity is measured using the fractal dimension box-counting algorithm;

[0155] The layout complexity is measured using the image two-dimensional entropy algorithm;

[0156] Color complexity was measured using the color complexity measurement;

[0157] “View structure representation”: the view area vector is calculated using the view area calculation method;

[0158] The viewshed perimeter vector is measured using the viewshed perimeter calculation method;

[0159] The sightline compactness vector is calculated using the viewshed compactness calculation method;

[0160] The view occlusion vector is calculated using the view occlusion calculation method;

[0161] The viewshed skewness vector is calculated using the viewshed line of sight skewness calculation method;

[0162] The viewshed drift vector is calculated using the viewshed drift calculation method.

[0163] 19-dimensional visual feature vector, the specific quantization method is shown in the following table:

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170] like Figure 17 Shown is an overview of the “Visual Entity Representation” search parameters of the Street View search system;

[0171] like Figure 18 The following is an overview of the search parameters for "visual impression representation" of the Street View search system;

[0172] like Figure 19 The following is an overview of the search parameters for the Street View search system's "View Structure Representation" search.

[0173] like Figure 20 The following is an overview of the search parameters for the Street View POI function of the Street View search system.

[0174] The technical principles of the present invention have been described above with reference to specific embodiments. These descriptions are intended solely to illustrate the principles of the present invention and are not to be construed in any way as limiting the scope of protection of the present invention. Based on the explanations herein, those skilled in the art will readily conceive of other specific embodiments of the present invention without inventive effort, and such embodiments will fall within the scope of protection of the present invention.

Claims

1. A city street view information retrieval method, characterized in that: The following steps are involved: S100, street view data acquisition, obtaining a city's entire street view image information and its corresponding geographic longitude and latitude information through the Google Maps API interface; S200, Street View Image Visual Feature Extraction and Quantification, performs multi-dimensional extraction and quantitative analysis of the visual features of street view images to construct a comprehensive visual representation system, including three categories: visual entity representation, visual impression representation, and visual field structure representation; S300: Comprehensive training of street view data and formation of an intelligent search system. This involves using a self-organizing map neural network algorithm to comprehensively train the street view visual representation system dataset, constructing a structured and visual street view feature distribution map, and forming the core system of the intelligent search system. S400, the use of intelligent search system, establishes a search system with visual feature data set as the core, and provides intuitive and efficient urban street view data query and analysis services through street view visual information retrieval and urban street view panorama display and interactive exploration.

2. The urban street view information retrieval method according to claim 1, characterized in that: In step S200, the three categories of visual representations are subdivided into 19-dimensional visual feature vectors based on the characteristics of the street view image, specifically including: Visual entity representation: Building structure, used to detect the visual proportion of artificial buildings in street view images; Street Furniture, which detects the visual proportion of street furniture infrastructure in street view imagery; Openness, used to detect the visual proportion of sky in street view images; Green view rate, used to detect the visual ratio of vegetation in street view images; Walkable roads, used to detect the visual ratio of walkable roads in street view images; Roadway, used to detect the visual ratio of walkable roads in street view images; Multi-purpose sites, which detects the visual proportion of sites that can be used for multi-purpose purposes in street view imagery; Human activity, used to detect the visual proportion of people in street view images; Traffic activity, used to detect the visual proportion of vehicles in street view images; Landscape, used to detect the visual proportion of landscape in street view images; Visual impression representation: Texture complexity, which is used to detect the edge complexity of all details in street view images; Layout complexity, which is used to detect the number of elements and information uncertainty of the basic composition of street view images; Color complexity, used to detect the richness of color composition in street view images; Visual field structure representation: Viewshed area is used to measure the visible space area from a certain viewpoint on the street and is related to the connectivity of the space; Viewshed perimeter, used to measure the visible viewshed boundary perimeter of a certain viewpoint on the street; Sightline compactness measures the compactness of the visual field at a certain viewpoint on the street, reflecting the shape attributes of the visible space and the continuity and consistency of the experience. The closer the visual field space is to a circle, the higher the visual field compactness. View occlusion measures the length of the edge of the view space that is not defined by entities at a certain viewpoint on the street, quantifies the degree of influence of obstructions on the extension of vision, and reflects the visual transparency of the space. Viewshed skewness measures the degree of difference in radial sight length at a certain viewpoint on the street and quantifies the symmetry and uniformity of the viewshed space; View drift measures the distance and direction from a certain viewpoint on the street to the center point of its view space.

3. The urban street view information retrieval method according to claim 2, characterized in that: In step S300, the following steps are specifically included: S310, feature standardization of street view visual data. To improve the convergence of the model and the stability of the search results, the z-score algorithm is first applied to standardize all input parameters to ensure that each feature has a consistent weight within the same numerical range. The z-score formula is: Among them, x i is the i-th eigenvalue of the original data, μ is the data mean, and σ is the data standard deviation; S320, self-organizing map neural network algorithm is used to train the model. By training the input data with the self-organizing map neural network algorithm, the model can learn the intrinsic structure of the data, map high-dimensional data into a two-dimensional feature space, and retain the topological structure of the similarity relationship between the data.

4. The urban street view information retrieval method according to claim 3, characterized in that: Step S320 is specifically as follows: S321, initial network construction, setting the network node structure (30*30) of the self-organizing map neural network algorithm, and randomly initializing the weight vector of each node; S322, weight update. During the training process, the Euclidean distance between the input data and the network node is calculated to map the input data to the network node closest to it, and the weight vector of the node and its neighborhood is updated. The formula is: I BMU =argmin{||s-v k (t)||} v k (t+1)=v k (t)+α(t)h bk [s-v k (t)] Among them, I BMU It is the network node closest to the original data generated in each update process, v k (t) represents the updated weight at time t, s represents the original data, h bk is the Gaussian neighborhood function; S323, cluster formation, after multiple iterative training, similar street view data are clustered to adjacent network nodes to form a meaningful cluster distribution.

5. The urban street view information retrieval method according to claim 4, characterized in that: Based on the search system, the "addressing" search function can be expanded. The addressing search function is mainly used to quickly locate the geographical location of the street view in the city based on the user-defined street view visual features; users can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

6. The urban street view information retrieval method according to claim 4, characterized in that: Based on the search system, a "scene search" function can be expanded to help users understand the types and distribution characteristics of street scenes in the target area based on geographic location; users select the location of an area of ​​interest on the geographic map to view the types, quantity and characteristic distribution of street scenes contained in the area, thereby supporting rapid street view auditing and analysis.

7. The urban street view information retrieval method according to claim 4, characterized in that: Based on the search system, a "landscape relationship" search function can be expanded. The landscape relationship search function is used to establish a visual connection between the street view image and the geographic map, helping users to intuitively understand the spatial distribution of different types of street scenes in the city. Through color coding, the search system will display street scene types on both maps simultaneously, allowing users to understand the relationship between street scene characteristics and geographic location from a macro perspective. In the search system interface, street view types in the overall street view map will be assigned different color tags to distinguish different types of street views. Corresponding color tags will appear on the geographic map, clearly showing the distribution of these street view types in the city.

8. The urban street view information retrieval method according to claim 4, characterized in that: Based on the search system, the "scenery preference" search function can be expanded. The scene preference search function is a personalized street view selection tool implemented by the search system based on the user's personal interests. The user finds the street view type that meets his or her personal interests in the street view overview map and marks it as the preference center point; in this way, the user's street view interest point is determined, and this preference information is used as the central benchmark of the heat map. The system will automatically generate a color spectrum mark and display the heat distribution of the entire street view overview map based on the degree of similarity; the color spectrum gradually changes from warm to cold, indicating the degree of preference matching of the street view: the warm color area represents the street view that highly matches the user's preference, while the cold color area represents the street view that has a lower degree of matching with the preference.

9. The urban street view information retrieval method according to claim 2, characterized in that Constructing a "visual entity representation" that includes: building structure, street furniture, openness, green view ratio, walkable roads, roadways, multi-functional areas, human activities, traffic vitality, and landscape. The quantification method uses the calculation of pixel ratio, that is, the proportion of the feature in the total pixels of the entire image; Characterized by the construction of a "visual impression representation" that includes: Texture complexity is measured using the fractal dimension box-counting algorithm; layout complexity is measured using the image two-dimensional entropy algorithm; color complexity is measured using the color complexity measurement; Construct a "visual structure representation", which includes: The viewshed area is calculated using the viewshed area calculation method; the viewshed perimeter is calculated using the viewshed perimeter calculation method; the line of sight compactness is calculated using the viewshed compactness calculation method; the viewshed occlusion is calculated using the viewshed occlusion calculation method; the viewshed skewness is calculated using the viewshed line of sight skewness calculation method; the viewshed drift is calculated using the viewshed drift calculation method.

Images