A city street view information retrieval method

Abstract

A kind of urban street view information retrieval method, comprising the following steps: S100, street view data acquisition, by Google map API interface, obtain a city full city street view image information and its corresponding geographic latitude and longitude information;S200, street view image visual feature extraction and quantization, the visual feature of street view image is extracted and quantitatively analyzed multidimensionally, constructs comprehensive visual representation system, wherein including: visual entity representation, visual impression representation and visual field structure representation three big categories;S300, the formation of street view data comprehensive training and intelligent search system, the street view visual representation system data set is comprehensively trained using self-organizing mapping neural network algorithm, constructs structured, visual street view feature distribution diagram, forms the core system of intelligent search system;S400, the use of intelligent search system, including street view visual information retrieval, visual display and interactive exploration.Provide intuitive, efficient city street view data query and analysis service.

Description

Technical Field

[0001] This invention relates to the field of geographic information, and in particular to a method for retrieving urban street view information. Background Technology

[0002] Street view technology is an important extension of geographic information services. By combining street view imagery of real-world environments with geographic information technology, it provides users with a virtual street view browsing experience. Currently, street view technology has become a fundamental tool in fields such as urban navigation, virtual tourism, and urban landscape display. Its main technical processes include the collection of street view data, the integration with geographic information, and the interactive display of street view imagery.

[0003] With the accelerated development of smart cities, the demand for urban street view information has far exceeded simple virtual tours and visualization experiences. While existing street view applications can provide basic street visualization services, they still face significant technical bottlenecks and shortcomings in the following aspects: 1. Limited utilization of street view data: Existing street view applications mainly focus on the collection and static display of street view data, lacking in-depth analysis and intelligent utilization of this data, thus failing to meet the complex spatial analysis needs of smart cities; 2. Incomplete extraction of visual features: Existing technologies mostly rely on single features, making it difficult to systematically and comprehensively quantify the multidimensional visual features of street views, resulting in a one-sided representation of street view information.

[0004] Given these shortcomings in technology and application, there is an urgent need for a tool that can efficiently integrate, analyze, and display urban street view information to meet the multi-level needs of everyone from ordinary users to urban planners. Summary of the Invention

[0005] To address the aforementioned shortcomings, the present invention aims to propose a street view information retrieval method that combines image recognition and geographic information technology to provide users with personalized and customized street view retrieval and display services, thereby achieving comprehensive mining and efficient utilization of street view information.

[0006] To achieve this objective, the present invention adopts the following technical solution:

[0007] A method for retrieving urban street view information includes the following steps:

[0008] S100, Street View Data Acquisition: Through the Google Maps API interface, obtain street view image information of the entire city and its corresponding geographical latitude and longitude information;

[0009] S200, Visual Feature Extraction and Quantization of Street View Images: This involves multi-dimensional extraction and quantitative analysis of the visual features of street view images, and the construction of a comprehensive visual representation system, which includes three major categories: visual entity representation, visual impression representation, and visual field structure representation.

[0010] The S300 and street view data are integrated for training and forming an intelligent search system. The self-organizing mapping neural network algorithm is used to train the street view visual representation system dataset to construct a structured and visualized street view feature distribution map, forming the core system of the intelligent search system.

[0011] The S400 intelligent search system, built around a visual feature dataset, provides intuitive and efficient urban street view data query and analysis services through street view visual information retrieval and panoramic urban street view display and interactive exploration.

[0012] Preferably, in step S200, the three major categories of visual representations are further subdivided into 19-dimensional visual feature vectors based on the characteristics of the street view image, specifically including:

[0013] Visual entity representation:

[0014] Building structure is used to detect the visual proportion of man-made buildings in street view images;

[0015] Street furniture is used to detect the visual proportion of street furniture infrastructure in street view images;

[0016] Openness is used to detect the visual proportion of the sky in street view images;

[0017] Green view ratio is used to detect the visual proportion of vegetation in street view images;

[0018] Walkable roads are used to detect the visual proportion of walkable roads in street view images.

[0019] The driveway is used to detect the visual proportion of pedestrian walkways in street view images;

[0020] Multi-purpose sites are used to detect the visual proportion of sites in street view images that can be used for multiple purposes.

[0021] Human activity is used to detect the visual proportion of people in street view images;

[0022] Traffic vitality is used to detect the visual proportion of vehicles in street view images;

[0023] Landscape, used to detect the visual proportion of landscape in street view images;

[0024] Visual impression representation:

[0025] Texture complexity is used to detect the edge complexity of all details in a street view image.

[0026] Layout complexity is used to detect the quantity and information uncertainty of the basic components of a street view image.

[0027] Color complexity is used to detect the richness of color composition in street view images;

[0028] Visual field structure representation:

[0029] The visible area is used to measure the visible space area at a certain viewpoint on a street, and it is related to the connectivity of the space.

[0030] Perimeter of viewpoint is used to measure the perimeter of the visible viewpoint boundary of a street.

[0031] Visual compactness measures the compactness of the field of view at a certain point on the street. It reflects the shape attributes of the visible space and the continuity and consistency of the experience. The closer the field of view is to a circle, the higher the visual compactness.

[0032] Visual occlusion measures the length of the edge in the visual space of a street viewpoint that is not defined by an entity. It quantifies the degree of influence of occlusion on the expansion of the line of sight and reflects the visual transparency of the space.

[0033] Visual field skewness measures the degree of difference in radial line-of-sight length at a certain viewpoint on the street, and quantifies the symmetry and uniformity of the visual field space.

[0034] Viewpoint drift is a measure of the distance and direction from a viewpoint on a street to the center of its viewpoint space.

[0035] Preferably, step S300 specifically includes the following steps:

[0036] S310. Feature standardization of street view visual data: In order to improve the convergence of the model and the stability of the search results, the z-score algorithm is first applied to standardize all input parameters to ensure that each feature has a consistent weight within the same numerical range.

[0037] The z-score formula is:

[0038]

[0039] Where, x i Let be the i-th feature value of the original data, μ be the data mean, and σ be the data standard deviation;

[0040] The S320 model is trained using a self-organizing map neural network algorithm. By training the input data using this algorithm, the model can learn the intrinsic structure of the data, mapping high-dimensional data to a two-dimensional feature space while preserving the topological structure of similar relationships between the data.

[0041] Preferably, step S320 specifically includes:

[0042] S321. Initial network construction: Set the network node structure (30*30) of the self-organizing map neural network algorithm, and randomly initialize the weight vector of each node;

[0043] S322, Weight Update: During training, the input data is mapped to the nearest network node by calculating the Euclidean distance between the input data and the network nodes. The weight vector of that node and its neighborhood is then updated, using the following formula:

[0044] I BMU =argmin{||sv k (t)||}

[0045] v k (t+1)=v k (t)+α(t)h bk [sv k (t)]

[0046] Among them, I BMU It is the network node that is closest to the original data generated during each update process, v k (t) represents the updated weight at time t, s represents the original data, and h represents the updated weight. bk It is a Gaussian neighborhood function;

[0047] S323. Clustering: After multiple iterations of training, similar street view data are clustered onto adjacent network nodes, forming a meaningful cluster distribution.

[0048] Furthermore, based on the search system, an "address search" function can be developed. The address search function is mainly used to quickly locate the geographical location of a street view in the city based on user-defined street view visual features. Users can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

[0049] Furthermore, based on the search system, a "street view search" function can be developed. This function helps users understand the types and distribution characteristics of street views within a target area based on their geographic location. Users can select an area of ​​interest on the geographic map to view the types, quantities, and characteristic distribution of street views in that area, thereby supporting rapid street view auditing and analysis.

[0050] Furthermore, based on the search system, a "landscape relationship" search function can be developed. This function establishes a visual connection between the street view panorama and the geographic map, helping users intuitively understand the spatial distribution of different types of street views in the city. Through color marking, the search system displays street view types simultaneously on both maps, enabling users to gain a macro-level understanding of the relationship between street view features and geographical location.

[0051] In the search system interface, different color markers will be assigned to the street view types in the street view panorama to distinguish different types of street views. The corresponding color markers will appear on the geographic map, clearly showing the distribution of these street view types in the city.

[0052] Furthermore, based on the search system, a "street view preference" search function can be developed. This function is a personalized street view selection tool implemented by the search system based on the user's personal interests. Users find street view types that match their interests in the overall street view map and mark them as preference centers. This determines the user's street view interest points, and this preference information serves as the central reference of a heatmap. The system automatically generates a color spectrum, displaying the entire street view map according to the degree of similarity. The color spectrum gradually changes from warm to cool colors, indicating the degree of street view preference matching: warm-toned areas represent street views that highly match the user's preferences, while cool-toned areas represent street views with a lower degree of matching.

[0053] Furthermore, in the "visual entity features", building structure, street furniture, openness, green view rate, walkable roads, driveways, multi-functional spaces, human activities, traffic vitality, and landscape are all calculated using the pixel proportion method, that is, the proportion of the feature in the total number of pixels in the entire image.

[0054] Furthermore, in the "visual impression features" section, texture complexity is measured using the fractal dimension box-counting algorithm;

[0055] The layout complexity is calculated using the two-dimensional entropy algorithm of the image.

[0056] Color complexity is measured using color complexity measurement.

[0057] Furthermore, in the "view area structure features", the view area vector is calculated using the view area calculation method;

[0058] The view perimeter vector is calculated using the view perimeter calculation method;

[0059] The line-of-sight compactness vector is calculated using the field-of-sight compactness calculation method;

[0060] The view occlusion vector is calculated using the view occlusion calculation method;

[0061] The field of view skewness vector is calculated using the field of view line skewness calculation method;

[0062] The view drift vector is calculated using the view drift calculation method.

[0063] One of the technical solutions described above includes the following beneficial effects: Through the aforementioned steps, the solution utilizes machine learning and big data analytics to achieve comprehensive and rapid data collection of urban streetscapes; by integrating features from multiple dimensions such as visual entity representation, visual impression representation, and visual structure representation, it provides a more comprehensive and multi-layered visual semantic understanding of urban streetscapes; by utilizing self-organizing mapping neural network technology, it effectively identifies the inherent connections and potential patterns in streetscape big data, improving the intelligence of streetscape retrieval; and by constructing a search platform, it provides an intuitive visual interface and customized interactive query functions, enabling users to efficiently and conveniently obtain the streetscape information they need. Overall, the solution achieves in-depth mining and accurate retrieval of multi-dimensional, large-scale urban streetscape data, effectively improving the efficiency and scientific nature of urban governance, planning, and management. It has significant innovative value and wide applicability in multiple application scenarios such as intelligent transportation, tourism planning, business analysis, and sustainable urban development. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the overall process;

[0065] Figure 2 This is a schematic diagram of the model training process using the self-organizing map neural network algorithm;

[0066] Figure 3 This is a schematic diagram illustrating the specific process of forming an intelligent search system;

[0067] Figure 4 This is a diagram illustrating the corresponding index relationship between visual parameters, street view image, street view functions, and geographic information functions.

[0068] Figure 5 It is the user interface of the intelligent search system;

[0069] Figures 6-7 It uses the addressing search function to perform street view search and addressing based on visual parameters;

[0070] Figures 8-9 It's a custom street view addressing feature;

[0071] Figures 10-11 It is street view addressing based on POI functionality;

[0072] Figure 12-13 It is a scene-finding function;

[0073] Figure 14 It's about the relationship between the landscape and the terrain;

[0074] Figure 15-16 It's a preference for scenic views;

[0075] Figure 17 This is an overview of the "visual entity representation" search parameters of the Street View search system;

[0076] Figure 18 This is an overview of the "visual impression representation" search parameters of the Street View search system;

[0077] Figure 19 This is an overview of the "view structure representation" search parameters of the Street View search system;

[0078] Figure 20 This is an overview of the search parameters for the "Street View POI function" in the Street View search system; Detailed Implementation

[0079] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0080] like Figure 1 As shown, a method for retrieving urban street view information includes the following steps:

[0081] S100, Street View Data Acquisition: Through the Google Maps API interface, obtain street view image information of the entire city and its corresponding geographical latitude and longitude information;

[0082] S200, Visual Feature Extraction and Quantization of Street View Images: This involves multi-dimensional extraction and quantitative analysis of the visual features of street view images, and the construction of a comprehensive visual representation system, which includes three major categories: visual entity representation, visual impression representation, and visual field structure representation.

[0083] The S300 and street view data are integrated for training and forming an intelligent search system. The self-organizing mapping neural network algorithm is used to train the street view visual representation system dataset to construct a structured and visualized street view feature distribution map, forming the core system of the intelligent search system.

[0084] The S400 intelligent search system, built around a visual feature dataset, provides intuitive and efficient urban street view data query and analysis services through street view visual information retrieval and panoramic urban street view display and interactive exploration.

[0085] The solution utilizes machine learning and big data analytics to achieve comprehensive and rapid data collection of urban streetscapes through the aforementioned steps. By integrating features from multiple dimensions, including visual impression representation, visual entity representation, and visual structure representation, it provides a more comprehensive and multi-layered visual semantic understanding of urban streetscapes. It leverages self-organizing map neural network technology to effectively identify the inherent relationships and potential patterns in streetscape big data, improving the intelligence of streetscape retrieval. Finally, by constructing a search platform, it provides an intuitive visual interface and customized interactive query functions, enabling users to efficiently and conveniently obtain the streetscape information they need.

[0086] Overall, the solution enables in-depth mining and accurate retrieval of multi-dimensional, large-scale urban street view data, effectively improving the efficiency and scientific nature of urban governance, planning and management. It has significant innovative value and wide applicability in multiple application scenarios such as intelligent transportation, tourism planning, business analysis and sustainable urban development.

[0087] The system employs an efficient street view data acquisition method, extracting urban street view images and related geographic information from the entire city by calling the API interface of existing map data. The street view acquisition parameters are as follows:

[0088] Data acquisition accuracy: Street view image data is acquired at 50-meter intervals to ensure the continuity and integrity of street coverage.

[0089] Image size: The resolution of the acquired image is set to 512×512 pixels to meet the needs of efficient storage and computing, while also ensuring image clarity.

[0090] Viewpoint settings:

[0091] Horizontal viewing direction (heading): Set to 0°, meaning the street view is directly in front of the road.

[0092] Vertical viewing angle (pitch): Set to 20°, tilting slightly upwards to capture the overall view of the ground and buildings.

[0093] Field of view (FOV): Set to 120° to ensure a large lateral field of view coverage and capture more street view information.

[0094] Based on the above parameters, 81,478 street view images of the city were obtained.

[0095] In step S200, the three major categories of visual representations are further subdivided into 19-dimensional visual feature vectors based on the characteristics of the street view image, specifically including:

[0096] Visual entity representation:

[0097] Building structure is used to detect the visual proportion of man-made buildings in street view images;

[0098] Street furniture is used to detect the visual proportion of street furniture infrastructure in street view images;

[0099] Openness is used to detect the visual proportion of the sky in street view images;

[0100] Green view ratio is used to detect the visual proportion of vegetation in street view images;

[0101] Walkable roads are used to detect the visual proportion of walkable roads in street view images.

[0102] The driveway is used to detect the visual proportion of pedestrian walkways in street view images;

[0103] Multi-purpose sites are used to detect the visual proportion of sites in street view images that can be used for multiple purposes.

[0104] Human activity is used to detect the visual proportion of people in street view images;

[0105] Traffic vitality is used to detect the visual proportion of vehicles in street view images;

[0106] Landscape, used to detect the visual proportion of landscape in street view images;

[0107] Visual impression representation:

[0108] Texture complexity vectors are used to detect the edge complexity of all details in a street view image;

[0109] The layout complexity vector is used to detect the uncertainty of the number of elements and information in the basic composition of street scene images.

[0110] Color complexity vector is used to detect the richness of color composition in street view images;

[0111] Visual field structure representation:

[0112] The visible area vector is used to measure the visible space area at a viewpoint on a street and is related to the connectivity of the space.

[0113] The view perimeter vector is used to calculate the perimeter of the visible view boundary of a street viewpoint.

[0114] The visual compactness vector measures the compactness of the visual field at a certain viewpoint on the street, reflecting the shape attributes of the visible space and the continuity and consistency of the experience. The closer the visual field space is to a circle, the higher the visual compactness.

[0115] The visual occlusion vector measures the length of the edge not defined by an entity in the visual space of a street viewpoint, quantifies the degree of influence of occlusion on the expansion of the line of sight, and reflects the visual permeability of the space.

[0116] The field of view skewness vector measures the degree of difference in radial line-of-sight length at a certain viewpoint on the street, and quantifies the symmetry and uniformity of the field of view space.

[0117] The field of view drift vector measures the distance and direction from a viewpoint on the street to the center point of its field of view.

[0118] Based on the digital representation framework of the above street view visual information, search parameters for street views were formed. These parameters control the weight and magnitude of each street view visual feature by adjusting their values, providing a precise data foundation for intelligent retrieval and personalized services. Based on the comprehensive digital representation framework of the aforementioned 19-dimensional street view visual information, a more comprehensive multi-dimensional parameter system for urban street views was innovatively constructed. These multi-dimensional features not only meticulously cover multiple dimensions of visual entities, visual impressions, and visual field structure, but also achieve flexible control over the weight of each type of visual feature by precisely adjusting the values ​​of each feature. This framework can capture and distinguish richer and more complex street view details, enhancing the accuracy and flexibility of information retrieval and achieving a more precise and personalized query experience. Furthermore, this detailed feature decomposition lays a solid data foundation for subsequent intelligent analysis and applications, such as urban planning optimization and environmental assessment.

[0119] like Figure 2 As shown, step S300 specifically includes the following steps:

[0120] S310. Feature standardization of street view visual data: In order to improve the convergence of the model and the stability of the search results, the z-score algorithm is first applied to standardize all input parameters to ensure that each feature has a consistent weight within the same numerical range.

[0121] The z-score formula is:

[0122]

[0123] Where, x i Let be the i-th feature value of the original data, μ be the data mean, and σ be the data standard deviation;

[0124] The S320 model is trained using a self-organizing map neural network algorithm. By training the input data using this algorithm, the model can learn the intrinsic structure of the data, mapping high-dimensional data to a two-dimensional feature space while preserving the topological structure of similar relationships between the data.

[0125] This topological feature distribution map can effectively identify the inherent correlation and grouping characteristics of different street view data, reflect the visual distribution pattern of street view features, facilitate the efficient operation of subsequent retrieval and analysis modules, and provide more structured and hierarchical feature support for intelligent search systems.

[0126] like Figure 3 As shown, step S320 specifically involves:

[0127] S321. Initial network construction: Set the network node structure (30*30) of the self-organizing map neural network algorithm, and randomly initialize the weight vector of each node;

[0128] S322, Weight Update: During training, the input data is mapped to the nearest network node by calculating the Euclidean distance between the input data and the network nodes. The weight vector of that node and its neighborhood is then updated, using the following formula:

[0129] I BMU =argmin{||sv k (t)||}

[0130] v k (t+1)=v k (t)+α(t)h bk [sv k (t)]

[0131] Among them, I BMU It is the network node that is closest to the original data generated during each update process, v k (t) represents the updated weight at time t, s represents the original data, and h represents the updated weight. bk It is a Gaussian neighborhood function;

[0132] S323. Clustering: After multiple iterations of training, similar street view data are clustered onto adjacent network nodes, forming a meaningful cluster distribution.

[0133] like Figure 4As shown, through the aforementioned data training process, the Self-Organizing Map Neural Network (SON) algorithm connects the 19-dimensional visual parameters with their corresponding street view images. Combining the SON algorithm data with the street view's geographic information data, an index relationship is established between "visual parameters," "street view images," "street view functions," and "geographical information." Furthermore, the comprehensive street view map formed by the SON algorithm reveals the essential logic and correlation of street view visuals, establishing clear structured connections between visual types, clustering relationships, and neighborhood features, providing data-driven support for exploring potential street view patterns. Therefore, a street view search system is formed. In summary, the advantages of using the SON algorithm lie in its powerful unsupervised learning ability and topology preservation characteristics, enabling high-dimensional data to be effectively expressed and organized in a low-dimensional space. It can integrate complex and fragmented street view information into a feature distribution map with clear logical relationships and an intuitive structure. This feature distribution not only intuitively expresses the similarities and differences between street view visual features but also reveals the inherent patterns and logic of street view images through clustering and visualization, providing a new structured perspective for understanding and analyzing street view information. This provides effective support for further analysis of street view evolution patterns and distribution patterns, demonstrating unique advantages, especially in exploring urban spatial design, environmental optimization, and functional zoning. More importantly, by integrating street view visual information, a multi-level index relationship was constructed between "visual parameters," "street view image," "street view function," and "geographic information." This not only provides clear type classification and neighborhood feature revelation for street view data but also enables the system to achieve accurate multi-angle and multi-level queries and dynamic evaluation of street views in subsequent retrieval and analysis stages. This index system provides data-driven support for street view pattern mining and potential relationship exploration, and also lays a solid technical foundation for building an intelligent and personalized street view search platform. Through this series of data preprocessing and clustering processes, this step ultimately significantly improved street view data in terms of type identification, group division, and visualization, providing strong support for the efficiency and accuracy of urban street view information retrieval.

[0134] like Figure 5 The image shown is the interface of the street view search system.

[0135] like Figures 6-7 As shown, the "addressing" function in the street view search system is mainly used to quickly locate the geographical location of a street view in the city based on user-defined street view visual features. Users can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

[0136] Users can select and adjust various visual feature parameters in the left panel, such as building structure, street furniture, and green view rate. These parameters provide a search basis for addressing, and users can freely combine features to precisely define the type of street view they are interested in. For example, users can set high green view rate and high openness to locate areas with dense natural landscapes, or set low green view rate and high building density to find urbanized areas. Based on the visual feature parameters set by the user, the system will highlight street views that meet the criteria in the "Street View Panorama". Clicking the "Address" button will display the actual distribution of these street views in the city on the "Geographic Map". This function is applicable to multiple application scenarios, such as: Urban planning and design: Urban planners can use the addressing function to filter areas with specific street view characteristics based on the set visual features. Real estate and commercial site selection analysis: Real estate developers and merchants can use the addressing function to select sites based on the set street view feature parameters. Retailers can choose street view areas with high pedestrian traffic as potential site selection references to increase the exposure and foot traffic of their shops.

[0137] like Figures 8-9 As shown, this demonstrates the addressing functionality for a custom street view:

[0138] Users can manually select specific combinations of visual features in the "Street View Panorama" and click the "Search" button. The system will then mark the locations of matching street views on the "Geographic Map." This feature supports various personalized needs, such as: Outdoor activity preference filtering: Outdoor sports enthusiasts can select street views of open public spaces, such as squares or parks. The system will then display the urban distribution of these locations to help users find suitable areas for exercise. Tourist interest point search: Tourists can select street views with high green visibility, moderate building structures, and distinctive street furniture features to easily find areas with rich natural landscapes and suitable for relaxation within the city, providing a more enjoyable urban experience.

[0139] like Figures 10-11 As shown, street view addressing based on POI functionality:

[0140] Street view location using the POI (Point of Interest) function on the left side of the interface helps users accurately locate urban areas that meet specific needs based on both visual features and functional attributes. Users can first filter street views with specific visual characteristics by adjusting visual parameters, or directly select specific types of street views from the overall street view map. The system will then mark the distribution of these street views across the city on a geographic map based on the filtering criteria. Next, users can further select specific functional attributes (such as catering services, shopping services, financial services, etc.) as secondary filtering criteria. The system will automatically filter out street views that do not meet the functional attributes, ultimately presenting a distribution of street views that meet both visual characteristics and specific functions. This function not only provides users with multi-level filtering paths but also significantly improves the accuracy and depth of urban spatial exploration. Whether used for urban planning, functional area analysis, or finding street view locations with specific functions within a specific area, users can obtain intuitive and detailed results through this combined filtering mechanism, meeting complex urban research and application needs.

[0141] Operation steps of the address search function: 1. Enter the search system: Users select Chinese or English interface through the platform's start page, enter the overview page, understand the platform's functions, and then enter the main operation page; 2. Visual parameter settings: Users select "Visual Impression Representation" in the visual parameter adjuster, check and adjust the "Texture Density", "Layout Complexity" and "Color Richness" parameters, setting them to "High", "Medium" and "High" respectively; 3. Street view search: Based on the parameter settings, the platform immediately highlights street views that match the above visual characteristics in the overall street view map. Users click the "Address" button to locate the actual location of these street views on the geographic map; 4. Result analysis: Users can see that street views with high texture density, high color richness, and medium layout complexity are mainly concentrated in the city's commercial and tourist areas.

[0142] like Figures 12-13 As shown, based on the search system, a "street view search" function can be developed. The street view search function helps users understand the types and distribution characteristics of street views within a target area based on their geographical location. Users can select the area of ​​interest and view the types, quantities, and characteristic distribution of street views contained in that area, thereby supporting rapid street view auditing and analysis.

[0143] The operation steps of the "Streetscape Search" function are as follows: 1. Area selection: Users select an area of ​​the city on the geographic map using the polyline tool, such as the area around the city's Central Park; 2. Heat map generation: After clicking the "Streetscape Search" button, the platform generates a heat map on the overall street view map, showing the composition types and quantities of streetscapes in the selected area; 3. Result analysis: The heat map shows that the area around Central Park has a high density of streetscape types with high green visibility and high openness, indicating that the area has a high green coverage rate and is suitable for leisure and outdoor activities.

[0144] like Figure 14 As shown, based on the search system, a "landscape relationship" search function can be extended. The landscape relationship search function is used to establish a visual connection between the street view panorama and the geographic map, helping users to intuitively understand the spatial distribution of different types of street views in the city. Through color marking, the search system displays the street view types simultaneously on the two maps, enabling users to gain a macro-level understanding of the relationship between street view characteristics and geographical location.

[0145] In the search system interface, different color markers will be assigned to the street view types in the street view panorama to distinguish different types of street views. The corresponding color markers will appear on the geographic map, clearly showing the distribution of these street view types in the city.

[0146] By synchronously displaying color-coded markers, users can quickly determine the main distribution areas of a particular street view type within a city. For example, green markers might represent areas with abundant greenery, while yellow markers represent high-density building areas. This feature is ideal for scenarios such as: Street view distribution characteristic analysis: helping urban planners analyze the layout of street views with different visual characteristics within a city, providing a reference for block design and regional planning. The "Landscape Relationships" feature establishes a visual connection between the overall street view map and the geographic map, enabling users to explore street view distribution characteristics within a broader urban context, providing crucial platform support for urban design, landscape analysis, and spatial planning.

[0147] The operation steps of the landscape relationship function are as follows: 1. Color mark setting: When the user clicks the "Landscape Relationship" button, the platform assigns different color marks to different street view types on the street view panorama map and geographic map; 2. Spatial distribution analysis: Through the synchronized display of color marks, the user can quickly identify areas with abundant greenery (green marks) and high-density building areas (gray marks); 3. Result analysis: Users can intuitively see the spatial distribution of green areas and densely built-up areas in the city, providing a reference for urban planning and landscape design.

[0148] like Figures 15-16 As shown, based on the search system, a "street view preference" search function can be developed. This function is a personalized street view selection tool implemented by the search system based on the user's personal interests. Users find street view types that match their interests within the search system and mark them as preference centers. This determines the user's street view interest points, and this preference information serves as the central reference for a heatmap. The system automatically generates a color spectrum, displaying the entire street view map according to the degree of similarity. The color spectrum gradually changes from warm to cool colors, representing the degree of street view preference matching: warm-toned areas represent street views that highly match the user's preferences, while cool-toned areas represent street views with a lower degree of matching.

[0149] Through color-coding visualization, users can quickly identify urban areas that match their personal preferences. In the "Geographic Map," the system displays street view areas that match user preferences as colored dots on the city map. This preference mapping allows users to more intuitively locate and identify areas that meet their preferences in geographic space, facilitating the selection of suitable street view environments for actual urban exploration and planning.

[0150] This feature helps users quickly identify areas that match their personal preferences across a wide range of street view locations, providing efficient support for urban exploration, analysis, and personalization needs.

[0151] The steps for using the "Street View Preferences" feature are as follows: 1. Preference Setting: Users find street view types that match their personal interests in the overall street view map, such as park areas with high green visibility and high openness, and mark them as the center point of their preference; 2. Thermal Chromatography Generation: After clicking the "Street View Preferences" button, the platform generates a thermal chromatogram based on the user's preference settings, displaying street view areas that match the user's preferences; 3. Result Analysis: Users can see that street view areas that highly match their personal preferences are displayed in warm tones (yellow) in the overall street view map, and highlighted as colored dots on the geographic map, making it easy for users to quickly identify and access them.

[0152] In addition, the 19-dimensional visual features include:

[0153] “Visual entity features”: Building structure, street furniture, openness, green view rate, walkable roads, driveways, multi-functional spaces, human activities, traffic vitality, and landscape are all calculated using the pixel proportion method, that is, the proportion of the feature in the total number of pixels in the entire image.

[0154] “Visual Impression Representation”: Texture complexity is measured using the fractal dimension box-counting algorithm;

[0155] The layout complexity is calculated using the two-dimensional entropy algorithm of the image.

[0156] Color complexity is measured using color complexity measurement.

[0157] "Viewfield structure representation": The viewfield area vector is calculated using the viewfield area calculation method;

[0158] The view perimeter vector is calculated using the view perimeter calculation method;

[0159] The line-of-sight compactness vector is calculated using the field-of-sight compactness calculation method;

[0160] The view occlusion vector is calculated using the view occlusion calculation method;

[0161] The field of view skewness vector is calculated using the field of view line skewness calculation method;

[0162] The view drift vector is calculated using the view drift calculation method.

[0163] The 19-dimensional visual feature vectors, and the specific quantization methods are shown in the table below:

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170] like Figure 17 The image shows an overview of the "visual entity representation" search parameters of the Street View search system.

[0171] like Figure 18 The image shows an overview of the "visual impression representation" search parameters of the Street View search system.

[0172] like Figure 19 The image shows an overview of the "view structure representation" search parameters of the Street View search system.

[0173] like Figure 20 The image shows an overview of the search parameters for the "Street View POI Function" in the Street View search system.

[0174] The technical principles of the present invention have been described above with reference to specific embodiments. These descriptions are merely for explaining the principles of the invention and should not be construed as limiting the scope of protection of the invention in any way. Based on this explanation, those skilled in the art can readily conceive of other specific embodiments of the invention without inventive effort, and these embodiments will all fall within the scope of protection of the present invention.

Claims

1. A method of retrieving urban street scene information, characterized by, The method comprises the following steps: S100, street view data acquisition, acquiring city-wide street view image information and corresponding geographic latitude and longitude information through a Google Map API interface; S200, street view image visual feature extraction and quantification, multi-dimensional extraction and quantitative analysis of the visual features of the street view image are performed to construct a comprehensive visual representation system, which includes three categories: visual entity representation, visual impression representation, and visual field structure representation; S300, comprehensive training of street view data and formation of an intelligent search system, a self-organizing mapping neural network algorithm is used to comprehensively train the street view visual representation system data set, a structured and visual street view feature distribution map is constructed, and a core system of an intelligent search system is formed; S400, use of the intelligent search system, a search system is established based on the visual feature data set, and a direct and efficient city street view data query and analysis service is provided in the form of street view visual information retrieval and city street view panorama display and interactive exploration; In step S300, the following steps are included: S310, feature standardization of street view visual data, in order to improve the convergence of the model and the stability of the search results, z-score algorithm is applied to standardize all input parameters, so as to ensure that each feature has consistent weight in the same numerical range; The z-score formula is: ; in, For the first part of the original data 1 eigenvalue, The mean of the data. The standard deviation of the data; S320, model training by a self-organizing mapping neural network algorithm, the model can learn the internal structure of the data by training the input data through the self-organizing mapping neural network algorithm, and map the high-dimensional data to a two-dimensional feature space while preserving the topological structure of the similar relationships between the data; Step S320 is specifically: S321, initial network construction, the network node structure (30*30) of the self-organizing mapping neural network algorithm is set, and the weight vector of each node is randomly initialized; S322, weight update, in the training process, the input data is mapped to the network node closest to it by calculating the Euclidean distance between the input data and the network node, and the weight vector of the node and its neighborhood is updated, the formula is: ； ； wherein, is the network node that is closest to the original data produced in each update process, represents the update weight at time t, represents the original data, is a Gaussian neighborhood function; S323, clustering formation, after multiple iterations of training, similar street view data is clustered on adjacent network nodes to form a clustering distribution; "Visual entity representation" is constructed, which includes building construction, street furniture, openness, green view rate, walkable road, driveway, multi-functional site, human activity, traffic vitality, and landscape; the quantification method adopts pixel ratio calculation, that is, the proportion of the feature in the total image pixels; The feature is characterized by constructing "visual impression representation", which includes: Texture complexity, using fractal dimension Layout complexity, using image two-dimensional entropy algorithm to measure; color complexity, using color complexity measurement to measure; "Visual field structure representation" is constructed, which includes: The visual field area is calculated by the visual field area calculation method; the visual field perimeter is calculated by the visual field perimeter calculation method; the visual line compactness is calculated by the visual field compactness calculation method; the visual field occlusion degree is calculated by the visual field occlusion degree calculation method; the visual field skewness is calculated by the visual field line skewness calculation method; and the visual field drift degree is calculated by the visual field drift degree calculation method.

2. The urban street view information retrieval method of claim 1, wherein, In step S200, the three categories of visual representation are further divided into 19 visual feature vectors according to the characteristics of the street view pictures, including: Visual entity representation: Building structure, for detecting the visual proportion of artificial buildings in the street view image; Street furniture, for detecting the visual proportion of street furniture infrastructure in the street view image; Openness, for detecting the visual proportion of the sky in the street view image; Green view rate, for detecting the visual proportion of vegetation in the street view image; Walkable road, for detecting the visual proportion of roads available for walking in the street view image; Carriageway, for detecting the visual proportion of roads available for walking in the street view image; Multi-purpose site, for detecting the visual proportion of sites available for multi-purpose use in the street view image; Human activity, for detecting the visual proportion of people in the street view image; Traffic vitality, for detecting the visual proportion of vehicles in the street view image; Landscape, for detecting the visual proportion of landscape in the street view image; Visual impression representation: Texture complexity, for detecting the edge complexity of all details in the street view image; Layout complexity, for detecting the element quantity and information uncertainty of the basic combination of the street view image; Color complexity, for detecting the richness of color composition in the street view image; Visual field structure representation: Visual field area, for calculating the visible space area of a certain viewpoint of the street, related to the connectivity of the space; Visual field perimeter, for calculating the visible visual field boundary perimeter of a certain viewpoint of the street; Visual line compactness, for calculating the visual field compactness of a certain viewpoint of the street, reflecting the shape attribute and experience continuity of the visible space, the closer the visual field space to the circle, the higher the visual field compactness; Visual field occlusion degree, for calculating the edge length of the visual field space of a certain viewpoint of the street which is not defined by entities, quantifying the influence degree of the occlusion on the visual line extension in the space, reflecting the visual permeability of the space; Visual field skewness, for calculating the difference degree of the radial visual line length of a certain viewpoint of the street, quantifying the symmetry and uniformity of the visual field space; Visual field drift degree, for calculating the distance and direction from a certain viewpoint of the street to the center point of its visual field space.

3. The urban street view information retrieval method of claim 1, wherein, Based on the search system, the "addressing" search function can be expanded, which is mainly used to quickly locate the geographical position of the user-defined street view visual features in the city; the user can quickly locate the geographical distribution of the target street view by adjusting the visual feature parameters or directly selecting a specific street view.

4. The urban street view information retrieval method of claim 1, wherein, Based on the search system, the "scene" search function can be expanded, which helps users understand the street view types and distribution characteristics in the target area based on the geographical position; the user can view the street view types, quantities and feature distribution in the area by selecting the area of interest in the geographical map, thereby supporting rapid street view audit and analysis.

5. The urban street view information retrieval method of claim 1, wherein, Based on the search system, the "landscape relationship" search function can be expanded, which is used to establish a visual correlation between the street view panorama and the geographical map, helping users intuitively understand the spatial distribution of different types of street views in the city; through color marking, the search system synchronously displays the street view types on the two maps, enabling users to understand the relationship between street view features and geographical location from a macro perspective; In the interface of the search system, different color markers will be assigned to the street view types in the street view panorama to distinguish different types of street views, and corresponding color markers will appear on the geographical map, clearly presenting the distribution of these street view types in the city.

6. The urban street view information retrieval method of claim 1, wherein, Based on the search system, the "landscape interest preference" search function can be expanded, which is a personalized street view selection tool implemented by the search system according to the personal interests of users. Users find street view types that meet their personal interests in the street view panorama and mark them as preferred center points. Based on this, the user's street view interest points are determined, and this preference information is used as the center reference of the heat map. The system will automatically generate a color spectrum marker and display the entire street view panorama based on the degree of similarity. The color spectrum gradually changes from warm to cold, representing the degree of preference matching of the street view: warm-toned areas represent street views that highly match the user's preferences, while cold-toned areas represent street views that have a lower degree of preference matching.

Images