Method for constructing city block with fused street view vision and related device

Abstract

The application discloses a kind of city block construction methods and related equipment of fusion street view vision, belong to city planning field, method includes obtaining the multimodal data of target block;Multi-modal data is processed in time sequence alignment, obtain after alignment multi-source space-time data, and extract the static attribute information of target block;Based on the change rule of point of interest attribute data on time axis of static attribute information and after alignment multi-source space-time data generates the dynamic evolution trend information of target block;Static attribute information, current time point of interest attribute data after alignment multi-source space-time data and geographic space correlation data are merged, and the fusion feature information of target block is obtained;Fusion feature information is decomposed into first class attribute information, which is integrated with dynamic evolution trend information, to obtain the dynamic basic representation information of target block, with the advantages of realizing the accurate space-time alignment of multi-modal data, effectively separating block static spatial structure and dynamic function evolution characteristics.

Description

Technical Field

[0001] This invention belongs to the technical field of urban planning, and specifically relates to a method and related equipment for constructing urban blocks that integrate street view visuals. Background Technology

[0002] As the core spatial unit of urban functions and residents' activities, the accurate modeling of urban blocks is crucial for urban planning, traffic management, and resource allocation. With the advancement of sensing and geographic information technologies, multimodal data such as panoramic street view images, points of interest, semantic attributes, and geographic proximity relationships are becoming increasingly abundant, providing multi-dimensional information sources for block perception.

[0003] However, most existing technologies focus on the static attributes of neighborhoods, such as functional classification or spatial morphology extraction based on a single modality (visual or semantic), lacking deep integration of multimodal data and ignoring the co-evolutionary relationships of different data sources over time; some studies, although employing time series analysis, have the following shortcomings: Multimodal time series data lacks precise alignment, making it difficult to form a unified evolution description; mixing fixed spatial structures with variable functional dynamics weakens the stability of the representation and obscures future evolution trends; feature fusion lacks independence constraints, resulting in high-dimensional redundancy and reducing interpretability and decision support capabilities.

[0004] In summary, existing urban block modeling techniques still have significant shortcomings in terms of the collaborative utilization of multimodal time-series data, consistent representation of spatiotemporal evolution, and the independence and interpretability of feature fusion. There is an urgent need to propose a scheme that can effectively integrate temporal collaborative relationships and decouple fixed and variable features. Summary of the Invention

[0005] This application provides a method and related equipment for constructing urban blocks by integrating street view vision, which can achieve accurate spatiotemporal alignment of multimodal data and effectively separate the static spatial structure and dynamic functional evolution characteristics of blocks.

[0006] On the one hand, embodiments of this application provide a method for constructing urban blocks by integrating street view vision, including: Acquire multimodal data of the target street at multiple consecutive historical time points, wherein the multimodal data includes street view panoramic image sequences, point of interest semantic attribute sequences, and geographic proximity relationship data; The multimodal data is subjected to time-series alignment processing to obtain aligned multi-source spatiotemporal data; Static attribute information of the target block is extracted from the aligned multi-source spatiotemporal data, wherein the static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time. Based on the changes in the static attribute information and the interest point attribute data of the aligned multi-source spatiotemporal data on the time axis, dynamic evolution trend information of the target block is generated, wherein the dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transitions in the future. The static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data are feature merged to obtain the fused feature information of the target block. Based on the preset attribute independence constraint, the fused feature information is decomposed into a first type of attribute information, wherein the first type of attribute information includes at least the spatial morphology dimension, the functional composition dimension, and the spatial location dimension. The dynamic evolution trend information is used as the second type of attribute information, and the second type of attribute information and the first type of attribute information are integrated to obtain the dynamic basic representation information of the target block.

[0007] Furthermore, the temporal alignment processing of the multimodal data includes: The street view panoramic image sequence, the point of interest semantic attribute sequence, and the geographic proximity relationship data are respectively mapped to a unified spatiotemporal coordinate system to obtain initial spatiotemporal alignment data; The initial spatiotemporal alignment data is sampled using a sliding window with a preset time window as the step size, and the image texture feature tensor, semantic label frequency vector and spatial adjacency matrix are extracted within each time window; Calculate the structural similarity index of the image texture feature tensor for adjacent time windows, and determine the scene mutation frame based on the fluctuation amplitude of the structural similarity index along the time axis; A baseline timeline is constructed based on the timestamps of the scene mutation frames. The semantic tag frequency vector is projected onto the reference time axis through nearest neighbor matching to obtain the corrected interest point time series data corresponding to the frame level of the street view panoramic image sequence. The spatial adjacency matrix is ​​linearly interpolated according to the time interval of the reference time axis to obtain a spatial adjacency matrix synchronized with the corrected interest point time series data, so as to form the aligned multi-source spatiotemporal data.

[0008] Furthermore, the step of generating dynamic evolution trend information of the target street based on the changing patterns of the static attribute information and the interest point attribute data of the aligned multi-source spatiotemporal data along the time axis includes: Expanding the sequence of semantic attributes of interest points in the aligned multi-source spatiotemporal data along the time axis yields the temporal evolution curve of the proportion of interest point categories. The static attribute information is used as a spatial constraint mask to perform spatial weighting on the temporal evolution curve to obtain the denoised evolution curve. Extract the attribute dwell time period and attribute transition critical point from the denoised evolution curve, and calculate the category entropy stability index within the attribute dwell time period and the gradient change magnitude of the attribute transition critical point within a preset range. Construct a Markov state transition probability matrix based on the category entropy stability index and the gradient change magnitude; The state pointers corresponding to probabilities greater than a preset transition threshold in the Markov state transition probability matrix are used as the dynamic evolution trend information, wherein the dynamic evolution trend information includes a potential transition direction probability vector and a corresponding expected transition time window.

[0009] Furthermore, the step of feature merging of the static attribute information, the current-moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial correlation data includes: Using the road topology skeleton data in the static attribute information as a spatial guidance graph, a graph attention network is constructed. The current time-based interest point attribute data is used as node features, and the geospatial association data is used as edge weights. Graph convolution operation is performed to obtain the interest point distribution features under spatial constraints. The street view facade outline baseline in the static attribute information is used as the discriminator constraint condition of the generative adversarial network. Modal consistency verification is performed on the distribution features of interest points under the spatial constraints, and abnormal interest point feature components that do not conform to the geometric constraints of the street view facade outline baseline are eliminated. The interest point distribution features that have passed the verification and the visual features of the current frame of the street view panoramic image sequence are fused together by cross-modal attention to obtain the fused feature information, wherein each feature dimension in the fused feature information carries a spatial association weight with the static attribute information.

[0010] Furthermore, the step of decomposing the fused feature information into a first type of attribute information based on a preset attribute independence constraint includes: Obtain the spatial association weights associated with the static attribute information in the fused feature information, and use the spatial association weights as the initial basis vectors for orthogonal projection; Based on the preset attribute independence constraint, an orthogonal decomposition objective function is constructed in Hilbert space, wherein the orthogonal decomposition objective function is used to minimize the mutual information between the spatial morphology dimension, functional composition dimension and spatial location dimension; The fused feature information is substituted into the orthogonal decomposition objective function for iterative projection. In each iteration, the spatial correlation weight is used as a regularization term to constrain the projection direction until the mutual information between each dimension is lower than the preset independence threshold and the iteration stops. The three orthogonal components obtained by projection at the point of stopping iteration are respectively used as the spatial morphology dimension, functional composition dimension, and spatial location dimension.

[0011] Furthermore, the integration of the second type of attribute information and the first type of attribute information includes: The potential transition direction probability vector in the second type of attribute information is used as the time driving factor, and the spatial location dimension in the first type of attribute information is used as the spatial carrying basis. Based on the conditional variational autoencoder, the conditional coding layer uses the spatial bearing substrate as the conditional variable to perform joint probabilistic coding on the potential transition direction probability vector and the expected transition time window to generate a spatiotemporally coupled latent vector. The spatiotemporal coupling latent vector, the spatial morphology dimension, and the functional composition dimension are concatenated to obtain the concatenated features. The splicing features are transformed into dynamic basic representation information by the decoder. The dynamic basic representation information is a four-dimensional tensor structure. The first and second dimensions correspond to the horizontal and vertical coordinates of the target block spatial grid, respectively. The third dimension corresponds to the prediction time window, and the fourth dimension corresponds to the attribute transition probability distribution of each spatial grid within the corresponding prediction time window.

[0012] Furthermore, the extraction of static attribute information of the target block includes: Principal component analysis is performed on the image texture feature tensor in the aligned multi-source spatiotemporal data along the time axis to obtain the eigenvalues ​​and eigenvectors corresponding to each principal component. Project the image texture feature vectors of each historical time node onto the feature vectors corresponding to each principal component to obtain the projection value sequence of each principal component; calculate the temporal variance or low-frequency energy ratio of the projection value sequence of each principal component along the time axis, and determine the principal components whose temporal variance is lower than a preset stability threshold or whose low-frequency energy ratio is higher than a preset ratio as time-stable principal components, and reconstruct the feature vectors corresponding to the time-stable principal components into a static texture base. Edge detection and line fitting are performed based on the static texture base to extract the street scene facade outline baseline; Semantic segmentation and road centerline refinement are performed based on the static texture base to extract the road topology skeleton data; The road topology skeleton data is used to provide spatial location anchors in the feature merging step of the static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data.

[0013] On the other hand, embodiments of this application provide an urban block construction device that integrates street view vision, the device comprising: The acquisition module is used to acquire multimodal data of the target street at multiple consecutive historical time points, wherein the multimodal data includes street view panoramic image sequences, interest point semantic attribute sequences, and geographic proximity relationship data; The alignment module is used to perform temporal alignment processing on the multimodal data to obtain aligned multi-source spatiotemporal data; An extraction module is used to extract static attribute information of the target block from the aligned multi-source spatiotemporal data, wherein the static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time. The generation module is used to generate dynamic evolution trend information of the target block based on the change pattern of the static attribute information and the interest point attribute data of the aligned multi-source spatiotemporal data on the time axis. The dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transition in the future. The merging module is used to merge the static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data to obtain the fused feature information of the target block. The decomposition module is used to decompose the fused feature information into a first type of attribute information based on a preset attribute independence constraint, wherein the first type of attribute information includes at least a spatial morphology dimension, a functional composition dimension, and a spatial location dimension. The integration module is used to take the dynamic evolution trend information as the second type of attribute information, and integrate the second type of attribute information and the first type of attribute information to obtain the dynamic basic representation information of the target block.

[0014] In another aspect, embodiments of this application provide an electronic device, the device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the urban block construction method that integrates street view vision as described in any one of the first aspects.

[0015] In another aspect, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the urban block construction method fused with street view vision as described in any one of the first aspects.

[0016] Compared with existing technologies, the urban block construction method and related equipment fused with street view vision in this application can form aligned multi-source spatiotemporal data under a unified time reference by performing temporal alignment processing on street view panoramic image sequences, interest point semantic attribute sequences, and geographic proximity relationship data from multiple consecutive historical time nodes. This fundamentally solves the problem of temporal misalignment caused by sampling frequency and phase differences between multimodal data. By extracting static attribute information that does not change with time, the fixed spatial structure and dynamic functional characteristics are decoupled, avoiding the problem of weakened representation stability caused by mixed processing of the two. Using this static attribute as a spatial constraint, combined with the change law of interest point attributes on the time axis, a dynamic evolution trend is generated. Information enables the modeling results to predict future attribute transition trends, overcoming the shortcomings of traditional methods that mask evolution trends. It merges static attributes, current point-of-interest attributes, and geospatial correlation data, and decomposes the fused features into first-class attributes such as spatial morphology, functional composition, and spatial location—mutually orthogonal—based on preset attribute independence constraints. This reduces high-dimensional redundancy and improves feature interpretability. Dynamic evolution trends are integrated with the first-class attributes as a second-class attribute to form dynamic basic representation information. This allows the street block model to stably depict spatial structure while proactively reflecting functional changes, thus addressing the shortcomings of existing technologies in multimodal temporal collaborative utilization, spatiotemporal consistent representation, and feature fusion independence. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings. Figure 1 This is a flowchart illustrating a method for constructing urban blocks that integrates street view visuals, as exemplified in this application. Figure 2 This is a structural block diagram illustrating an urban block construction device that integrates street view visuals in the embodiments of this application; Figure 3 This is a structural block diagram illustrating an electronic device in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To facilitate understanding of the embodiments of this application, further explanation and description will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application. In the drawings, the dimensions and relative dimensions of components may be exaggerated for clarity and / or descriptive purposes. When exemplary embodiments can be implemented differently, a specific process sequence may be performed in a different order than that described. For example, two consecutively described processes may be performed substantially simultaneously or in the reverse order of their description. Furthermore, the same reference numerals denote the same components.

[0020] The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that would be recognized by one of ordinary skill in the art.

[0021] Traditional urban block modeling techniques suffer from insufficient data alignment accuracy when processing multimodal time-series data, making it difficult to capture the spatiotemporal evolution patterns of blocks. Existing methods often mix the fixed spatial structure of blocks with dynamic functional evolution, which reduces the stability of block representation and obscures its future evolution trend.

[0022] To address the problems in the prior art, this application provides a method and related equipment for constructing urban blocks that integrate street view visuals.

[0023] The following section first introduces a method for constructing urban blocks that integrates street view visuals, as provided in the embodiments of this application.

[0024] Figure 1This illustration shows a flowchart of a method for constructing urban blocks by integrating street view vision, according to an embodiment of this application. Figure 1 As shown in the figure, an embodiment of this application provides a method for constructing urban blocks by integrating street view vision, which includes the following steps: S101. Obtain multimodal data of the target block at multiple consecutive historical time points. The multimodal data includes street view panoramic image sequences, interest point semantic attribute sequences, and geographic proximity relationship data.

[0025] In this embodiment, the street view panoramic image sequence can be generated by a vehicle driven by a driver and equipped with a panoramic camera, which takes pictures along the street at different times to form a series of image data; the semantic attribute sequence of points of interest can be generated by a person conducting on-site surveys of shops, facilities, etc. in the street and recording their categories, functions, and other information; the geographic proximity relationship data can be generated by a person manually marking the connection relationships between different points of interest based on the street map.

[0026] S102. Perform time-series alignment processing on the multimodal data to obtain aligned multi-source spatiotemporal data.

[0027] Specifically, the street view panoramic image sequence, point of interest semantic attribute sequence, and geographic proximity relation data are mapped to a unified spatiotemporal coordinate system to obtain initial spatiotemporal aligned data. Sliding window sampling is performed on the initial spatiotemporal aligned data with a preset time window step size to extract the image texture feature tensor, semantic tag frequency vector, and spatial adjacency matrix within each time window. The structural similarity index of the image texture feature tensor of adjacent time windows is calculated, and scene mutation frames are determined based on the fluctuation amplitude of the structural similarity index along the time axis. A reference time axis is constructed based on the timestamps of the scene mutation frames. The semantic tag frequency vector is projected onto the reference time axis through nearest neighbor matching to obtain corrected point of interest temporal data corresponding to the frame level of the street view panoramic image sequence. The spatial adjacency matrix is ​​linearly interpolated according to the time interval of the reference time axis to obtain a spatial adjacency matrix synchronized with the corrected point of interest temporal data, thus forming aligned multi-source spatiotemporal data.

[0028] In this embodiment, the spatial location of street view images—including the capture location and time, geographic coordinates and creation / update time of points of interest, and geographic boundaries and effective times of geographic proximity relationships—can be unified to a standard geographic coordinate system using georegistration techniques, such as GPS coordinates or high-precision map matching. Standard geographic coordinate systems include, but are not limited to, WGS84 and UTM. The timestamps are then calibrated to a unified time standard using a time synchronization protocol. Furthermore, a geographic information system-based projected coordinate system and a standard time synchronization protocol can be used to spatially transform and time-synchronize data from different sources.

[0029] The preset time window can be set as needed without specific limitations. For the image texture feature tensor, the output of the intermediate layer of a pre-trained convolutional neural network can be used as a feature extractor to extract features from the street view panoramic image sequence within each time window, resulting in a high-dimensional image feature representation. For the semantic label frequency vector, the predefined semantic categories of all points of interest within the time window can be statistically analyzed, such as the frequency or proportion of occurrence of dining, shopping, and education. For the spatial adjacency matrix, an adjacency matrix representing the connection relationships between different areas or points of interest within the block can be constructed based on the geographical proximity relationship data within the time window, where the element values ​​can represent distance, connectivity, or interaction strength. In addition, the image texture feature tensor can also utilize traditional image processing methods such as local binary mode, Gabor filters, or gray-level co-occurrence matrices to extract texture features from street view images. The semantic label frequency vector can also be constructed by extracting and classifying keywords from the text descriptions in the point of interest data, and statistically analyzing the occurrence frequency of each category. The spatial adjacency matrix can also be constructed based on the road network, building layout, or functional zoning within the block, defining nodes and edges. For example, nodes represent block units and points of interest, and edges represent road connections and spatial adjacency.

[0030] The structural similarity index measures the similarity between two images. It is calculated based on brightness, contrast, and structure. It works by calculating the structural similarity index between image frames corresponding to the image texture feature tensors of adjacent time windows, and setting a similarity threshold. When the structural similarity index is less than the threshold or its rate of change exceeds a preset threshold, a scene abrupt change is considered to have occurred. Besides the structural similarity index, mean squared error, peak signal-to-noise ratio, or feature distance based on deep learning can also be used (measures the difference between image texture feature tensors; by analyzing the fluctuation amplitude of these difference indicators along the time axis, for example, calculating their first or second derivatives, when the fluctuation amplitude exceeds a certain dynamic threshold, it can be identified as a scene abrupt change frame; the dynamic threshold is set as needed and is not specifically limited).

[0031] In this embodiment, the timestamps of all scene mutation frames are arranged chronologically to form a discrete set of time points. These time points are used as key nodes on the baseline timeline to represent important milestones in the evolution of the neighborhood. Furthermore, after identifying the scene mutation frames, these frames can be clustered or filtered to select the most representative mutation points as anchor points for the baseline timeline. For example, the anchor points can be selected based on the significance of the mutation magnitude or its duration, ensuring that the baseline timeline can capture the key stages of neighborhood evolution.

[0032] In this embodiment, for each time point on the baseline time axis, the semantic tag frequency vector with the closest timestamp in the original point of interest semantic attribute sequence is found, and this vector is used as the semantic information corresponding to that baseline time point. When multiple nearest semantic tag frequency vectors exist, the semantic tag frequency vector with the smallest time difference is selected. In addition to nearest neighbor matching, an average or weighted average method within a time window can also be used. That is, for each time point on the baseline time axis, a small neighboring time window is defined, and the semantic tag frequency vectors of all point of interest semantic attribute sequences within the window are averaged or weighted averaged, where the weights can be set according to the time distance to obtain the semantic information corresponding to that time point.

[0033] In this embodiment, for any two adjacent scene abrupt change frames on the reference time axis, the spatial adjacency matrix corresponding to these two abrupt change frames is linearly interpolated based on their relative positions on the time axis. For example, if a time point is located between T1 and T2, with a distance of d1 from T1 and a distance of d2 from T2, its spatial adjacency matrix can be calculated using the formula: M1×(d2 / (d1+d2))+M2×(d1 / (d1+d2)), where M1 is the spatial adjacency matrix at time point T1 and M2 is the spatial adjacency matrix at time point T2. Besides linear interpolation, spline interpolation, polynomial interpolation, or methods based on time series prediction models can also be used to estimate the missing spatial adjacency matrix on the reference time axis.

[0034] It should be noted that when the update time of the point of interest data is inconsistent with the scene change frame, the actual update time of the point of interest data can be used as an auxiliary anchor point to construct a composite reference time axis by weighted fusion, rather than a single visual reference axis.

[0035] In this embodiment, multimodal data is mapped to a unified spatiotemporal coordinate system. Image texture feature tensors, semantic tag frequency vectors, and spatial adjacency matrices are extracted through a sliding window to achieve data stream structuring. The structural similarity of the image texture feature tensors is used to determine scene mutation frames and construct a reference time axis. The semantic tag frequency vectors are projected onto the reference time axis through nearest neighbor matching, which solves the problem of misalignment with the timestamps of the image sequence. Linear interpolation is performed on the spatial adjacency matrix to ensure the temporal continuity of geographical relationships. Discrete heterogeneous data is transformed into aligned multi-source spatiotemporal data, providing a data foundation for static attribute extraction and dynamic evolution analysis.

[0036] S103. Extract the static attribute information of the target block from the aligned multi-source spatiotemporal data. The static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time.

[0037] Specifically, principal component analysis is performed on the image texture feature tensor of the aligned multi-source spatiotemporal data along the time axis to obtain the eigenvalues ​​and eigenvectors corresponding to each principal component; the image texture feature vectors of each historical time node are projected onto the eigenvectors corresponding to each principal component to obtain the projection value sequence of each principal component; the temporal variance or low-frequency energy proportion of the projection value sequence of each principal component along the time axis is calculated, and the principal components with a temporal variance lower than a preset stability threshold or a low-frequency energy proportion higher than a preset proportion are determined as temporally stable principal components, and the eigenvectors corresponding to the temporally stable principal components are reconstructed into a static texture base; edge detection and line fitting are performed based on the static texture base to extract the street scene facade outline baseline; semantic segmentation and road centerline refinement are performed based on the static texture base to extract road topology skeleton data; wherein, the road topology skeleton data is used to provide spatial location anchors in the feature merging step of static attribute information, current time point of interest attribute data of aligned multi-source spatiotemporal data, and geospatial association data.

[0038] Principal component analysis (PCA) can be implemented in various ways. In this embodiment, PCA can be performed using singular value decomposition (SVD). The image texture feature tensor of the time series is expanded into a data matrix, where each row represents the feature vector at a given time point. SVD decomposition is then performed on this data matrix to obtain the principal components. Alternatively, the covariance matrix of the image texture feature tensor on the time axis can be calculated, and eigenvalue decomposition can be performed on this covariance matrix to obtain eigenvalues ​​and eigenvectors.

[0039] Among them, the eigenvalues ​​are used to measure the overall variance explained by the principal components and can be used to determine the number of candidate principal components, but are not directly used to determine whether the principal components have temporal stability. For each candidate principal component, the image texture feature vectors of each historical time node are projected onto the feature vectors corresponding to the principal component to obtain the principal component projection value sequence. The temporal variance or low-frequency energy ratio of the principal component projection value sequence along the time axis is calculated, and the temporally stable principal components are selected according to the preset stability threshold. The stability threshold can be determined by historical sample statistics, cross-validation or empirical parameters. The image texture feature vectors are obtained by extracting the image texture feature tensor point by point along the spatial index.

[0040] Among them, the static texture base reflects the stable visual structure of the street, edge detection can use image processing algorithms such as Canny operator, Sobel operator or Laplacian operator, and line fitting can use Hough transform to identify line segments in the image.

[0041] In this embodiment, semantic segmentation can classify pixels in an image into different semantic categories, such as roads, buildings, and vegetation. Semantic segmentation can employ convolutional neural network-based models, such as FCN, PSPNet, or HRNet, to classify each pixel in the image. Road centerline thinning can utilize morphological thinning algorithms, such as the Zhang-Suen algorithm, or distance-transform-based methods. Furthermore, graph neural networks can be used to post-process the semantic segmentation results, representing the road region as a graph structure and generating road centerlines using graph algorithms, such as shortest path algorithms or topological skeleton extraction algorithms. The road topological skeleton data provides a stable, geographically meaningful reference frame, enabling accurate spatial alignment and fusion of data from different sources.

[0042] Before principal component analysis decomposition, time-frequency analysis, such as wavelet transform, can be performed on the image texture feature tensor on the time axis to separate low-frequency components (corresponding to stable structures) and high-frequency components (corresponding to dynamic changes). Then, principal component analysis is performed on the low-frequency components to extract static attributes.

[0043] In this embodiment, principal component analysis is performed on the image texture tensor to decouple dynamic fluctuations from static structures. Temporally invariant principal components are selected to reconstruct the static texture base to reflect the inherent spatial characteristics of the street. Based on this static texture base, edge detection and line fitting are performed, and the facade outline baseline is extracted as a geometric reference. The road topology skeleton is extracted through semantic segmentation and road centerline refinement, which serves as a spatial anchor point to guide the merging of multi-source features, thereby solving the problem of positioning deviation. Static attribute information supports the stability of spatial morphology, functional composition, and location dimension, and provides a spatial constraint mask for dynamic evolution, which to a certain extent improves the recognition accuracy of attribute dwell time and transition critical points.

[0044] S104. Based on the changing patterns of static attribute information and the interest point attribute data of aligned multi-source spatiotemporal data on the time axis, dynamic evolution trend information of the target block is generated. The dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transitions in the future.

[0045] Specifically, the semantic attribute sequence of interest points in the aligned multi-source spatiotemporal data is expanded along the time axis to obtain the temporal evolution curve of the proportion of interest point categories. The static attribute information is used as a spatial constraint mask to perform spatial weighting on the temporal evolution curve to obtain the denoised evolution curve. The attribute dwell time period and attribute transition critical point are extracted from the denoised evolution curve, and the category entropy stability index and the gradient change magnitude of the attribute transition critical point within the preset range are calculated. A Markov state transition probability matrix is ​​constructed based on the category entropy stability index and the gradient change magnitude. The state direction corresponding to the probability greater than the preset transition threshold in the Markov state transition probability matrix is ​​used as the dynamic evolution trend information, where the dynamic evolution trend information includes the potential transition direction probability vector and the corresponding expected transition time window.

[0046] In this embodiment, for each time point, the number of various points of interest (POIs) within the target block, such as restaurants, retail outlets, educational institutions, and residences, is counted, and their proportion among all POIs is calculated, thus forming a multidimensional vector. The change of this multidimensional vector over time constitutes the temporal evolution curve. Alternatively, a sliding time window approach can be used to aggregate the semantic attributes of POIs within each window, calculate the frequency distribution of each category, and use the center time point of the window as the timestamp of this distribution. Connecting the distributions of each timestamp yields the temporal evolution curve.

[0047] In this embodiment, static attribute information includes road topology, building outlines, and land parcel division. Static attribute information can be converted into a spatial mask, which assigns different weights to different spatial areas within the block. When performing spatial weighting on the temporal evolution curve, the proportion of interest point categories in different areas can be weighted averaged or weighted aggregated according to the degree of matching between the spatial distribution of interest points and the static attribute information, thereby weakening the impact of randomness or local anomalies. The weight values ​​are set as needed and are not specifically limited. For example, when the static attributes of a certain area indicate that its main function is residential, but a large number of commercial interest points suddenly appear in the time series data, the weight of these commercial interest points in the overall evolution curve can be reduced through a spatial constraint mask to reflect that they may be noise or short-term phenomena.

[0048] In this embodiment, change point detection algorithms in time series analysis, such as the CUSUM algorithm and statistical test-based methods, can be applied to identify moments in the denoised evolution curve where statistical characteristics change significantly. These moments are attribute transition critical points. The time period between two adjacent transition critical points is the attribute dwell period. During the attribute dwell period, the Shannon entropy of the interest point category distribution can be calculated, and its volatility during the period can be evaluated. For example, the standard deviation or coefficient of variation of the entropy value can be calculated as the category entropy stability index. The smaller the category entropy stability index, the more stable the functional structure. For attribute transition critical points, the Euclidean distance, cosine similarity change rate, or absolute change in the proportion of the dominant category can be calculated within a preset time window before and after the critical point. This can be used as the gradient change amplitude. The larger the amplitude, the more drastic the functional change. The preset time window can be set as needed and is not specifically limited.

[0049] In this embodiment, the functional state of a block during different attribute dwell times is defined as a discrete state in a Markov chain, such as a high residential density state, a mixed commercial state, or an industrial-dominated state. The category entropy stability index is used to evaluate the persistence of the current state, i.e., the self-transition probability. The higher the stability, the greater the self-transition probability. The gradient change magnitude is used to evaluate the possibility of transitioning from the current state to other states. The greater the gradient change magnitude, the higher the probability of a state transition. The direction of the gradient change is determined by the change in the distribution of interest point categories before and after the transition. In this embodiment, a Markov state transition probability matrix can be trained using the state sequence observed in historical data and the above-mentioned quantitative indicators, where each element represents the probability of transitioning from one state to another.

[0050] In this embodiment, for the current functional state of the block, the Markov state transition probability matrix can be queried to filter out all target states whose transition probabilities are greater than a preset transition threshold. These target states and their corresponding transition probabilities together constitute a potential transition direction probability vector, indicating the possible direction and probability of the block's future evolution. The preset transition threshold is set as needed and is not specifically limited. At the same time, by combining the duration of similar transition processes observed in historical data or by calculating the average residence time of the Markov chain, the expected transition time window required to reach these potential transition directions can be estimated.

[0051] In this embodiment, by transforming the semantic attribute sequence of points of interest into a temporal evolution curve and using static attribute information for spatial weighted denoising, the accuracy and stability of data representation are improved to a certain extent. By calculating the stability index of attribute residence time and the gradient change amplitude of transition critical points, the intrinsic mechanism of block function evolution is captured. By constructing a Markov state transition probability matrix, the block evolution process is modeled as a state transition, thereby predicting the potential future transition direction and expected time window, solving the problem of the difficulty in accurately predicting the trend of block evolution.

[0052] S105. The static attribute information, the current time point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial correlation data are merged to obtain the fused feature information of the target block.

[0053] Specifically, a graph attention network is constructed using the road topology skeleton data from the static attribute information as a spatial guidance graph. The current moment's interest point attribute data is used as node features, and geospatial association data is used as edge weights. A graph convolution operation is performed to obtain the interest point distribution features under spatial constraints. The street view facade outline baseline from the static attribute information is used as the discriminator constraint condition of the generative adversarial network (GAN) to perform modal consistency verification on the spatially constrained interest point distribution features, eliminating abnormal interest point feature components that do not conform to the geometric constraints of the street view facade outline baseline. The verified interest point distribution features and the current frame visual features of the street view panoramic image sequence are then fused through cross-modal attention to obtain fused feature information. Each feature dimension in the fused feature information carries spatial association weights with the static attribute information.

[0054] The road topology skeleton data is an abstract and simplified representation of the road network structure within the target block. For example, it can be obtained by vectorizing high-resolution remote sensing images after image processing, or extracted and simplified into centerlines and nodes from existing geographic information system data. In this graph attention network, the attribute data of the points of interest at the current moment is used as node features. The attribute data of the points of interest at the current moment can be descriptive information such as the category encoding of the points of interest, such as one-hot encoding or embedding vector, business type, rating, or passenger flow. At the same time, geospatial association data is used as edge weights. Geospatial association data can represent the spatial relationship strength between points of interest, such as the inverse Euclidean distance, road connectivity, or shared boundaries. By performing graph convolution operations, that is, aggregating and transforming the node features on the graph structure, the distribution features of points of interest under spatial constraints are obtained. The graph convolution operation can be implemented by spectral domain graph convolution or spatial domain graph convolution.

[0055] Among them, the street view facade outline baseline is the geometric boundary or outline of the building facade within the block. For example, it can be obtained by edge detection and line fitting of the street view panoramic image, or extracted from the three-dimensional building model and projected onto the two-dimensional plane. Modal consistency verification is to verify whether there is a conflict or inconsistency between different modal data, such as semantic interest point data and visual geometric data. In this embodiment, the discriminator can determine whether the spatial distribution implied by the interest point features matches the physical boundary defined by the street view facade outline baseline, thereby eliminating abnormal interest point feature components that do not conform to the geometric constraints of the street view facade outline baseline. These abnormal components may be caused by data acquisition errors, semantic annotation deviations, or inaccurate spatial mapping. For example, a certain interest point feature may semantically represent an entity, but its spatial location is obviously beyond the scope of the building facade.

[0056] The visual features of the current frame of the street view panoramic image sequence can be extracted from the street view image using pre-trained convolutional neural networks such as ResNet and VGG. These features contain rich texture, color, and semantic information. The cross-modal attention fusion mechanism allows the system to dynamically allocate weights based on the importance of different modal features. For example, a Transformer-based attention mechanism can be used to enable interest point features and visual features to pay attention to each other and extract complementary information. During the fusion process, it is ensured that each feature dimension in the fused feature information carries spatial association weights with static attribute information. In the fused feature vector, each component can trace the strength of its relationship with static spatial structures such as road topology skeleton or street view facade outline baseline, thereby enhancing the interpretability and spatial localization capability of the features.

[0057] In this embodiment, a graph attention network is constructed using road topology skeleton data to closely integrate interest point attribute data with the inherent spatial structure of the street block. This gives the extracted interest point distribution features spatial constraints, avoiding logical omissions caused by simple data stacking. The street view facade outline baseline is used as a discriminator constraint condition for the generative adversarial network, and modal consistency verification is performed to identify and eliminate abnormal interest point feature components that may be semantically reasonable but do not conform to reality in physical space. This ensures a high degree of consistency between semantic data and visual geometry. Through cross-modal attention fusion, the verified interest point distribution features are integrated with the visual features of the street view panoramic image, and each feature dimension is assigned a spatial correlation weight with static attribute information. This not only enriches the expressive power of the features but also gives the fused features clear spatial semantics and interpretability, providing basic data for subsequent urban street block attribute decomposition and dynamic evolution analysis.

[0058] S106. Based on the preset attribute independence constraint, the fused feature information is decomposed into the first type of attribute information, wherein the first type of attribute information includes at least the spatial morphology dimension, the functional composition dimension, and the spatial location dimension.

[0059] Specifically, the spatial association weights associated with static attribute information in the fused feature information are obtained, and these spatial association weights are used as the initial basis vectors for orthogonal projection. An orthogonal decomposition objective function is constructed in Hilbert space based on preset attribute independence constraints. This orthogonal decomposition objective function minimizes the mutual information between the spatial morphology dimension, functional composition dimension, and spatial location dimension. The fused feature information is substituted into the orthogonal decomposition objective function for iterative projection. In each iteration, the spatial association weights are used as a regularization term to constrain the projection direction until the mutual information between each dimension is lower than a preset independence threshold, at which point the iteration stops. The three orthogonal components obtained from the projection at the point of stopping iteration are then used as the spatial morphology dimension, functional composition dimension, and spatial location dimension, respectively.

[0060] Among them, the spatial correlation weights associated with static attribute information in the fused feature information are weight coefficients assigned to each feature dimension in the fused feature information through a cross-modal attention fusion mechanism during the feature merging process. These weights represent the spatial correlation between the feature dimension and the static attribute information. These weights can quantify the consistency or correlation between each component in the fused feature and the physical structure of the block. For example, these weights can be the attention scores output by the attention mechanism, reflecting the degree of attention each part of the fused feature pays to the static attribute information. Alternatively, they can be obtained by calculating the co-occurrence frequency or geometric matching degree between each dimension of the fused feature and the static attribute information in space. Using these spatial correlation weights as the initial basis vectors of orthogonal projection means that the starting direction of the decomposition process is guided to the direction that is strongly correlated with the inherent spatial structure of the block, providing a physically meaningful anchor point for subsequent feature separation. For example, the spatial correlation weight vector can be normalized and directly used as the initial basis vector, or several main spatial correlation directions can be extracted as the initial basis vector group by performing dimensionality reduction processing such as principal component analysis or non-negative matrix decomposition on the weight vector.

[0061] Preset attribute independence constraints are requirements for the independence between different attribute dimensions decomposed during feature decomposition. For example, a mutual information threshold can be set. When the mutual information between any two dimensions is less than the mutual information threshold, they are deemed to satisfy the independence constraint. The mutual information threshold can be set as needed and is not specifically limited. Hilbert space is a complete inner product space, which provides a mathematical basis for defining and optimizing objective functions, enabling the solution of complex optimization problems such as minimizing mutual information. For example, by using kernel functions to map data in the original feature space to a higher-dimensional Hilbert space, linearly separable or orthogonal decomposition can be achieved in a higher dimension.

[0062] The orthogonal decomposition objective function is a mathematical expression used to guide the feature decomposition process. Its purpose is to decompose the original fused feature information into mutually independent dimensions while satisfying orthogonality or independence constraints. The objective function includes a minimization term such as mutual information and a constraint term such as orthogonality constraint. For example, the objective function is: min(mutual information (dimension 1, dimension 2) + mutual information (dimension 1, dimension 3) + mutual information (dimension 2, dimension 3)). At the same time, the orthogonality constraint is added to minimize the mutual information between the spatial morphology dimension, functional composition dimension, and spatial location dimension. This means that these three dimensions should be as statistically independent as possible, that is, the information of one dimension should not contain or predict the information of another dimension. For example, the mutual information is estimated by calculating the joint probability distribution and marginal probability distribution of the feature vectors of each dimension and used as the optimization objective.

[0063] Iterative projection is an optimization algorithm that gradually decomposes the original feature information into the target dimension by repeatedly performing projection operations until the preset conditions are met. In each iteration, the algorithm adjusts the projection direction and projection results according to the objective function and constraints. For example, it may employ optimization strategies such as gradient descent or alternating least squares. In each iteration, the gradient of the objective function is calculated, and the projection matrix is ​​updated along the gradient direction. Spatial correlation weights are used as regularization terms to constrain the projection direction. This is to ensure that the decomposed dimensions not only meet the independence requirements but also maintain their correlation with the inherent spatial structure of the block during the iterative decomposition process. The regularization term penalizes the projection direction, guiding the projection process towards directions that are highly consistent with the spatial correlation weights. For example, a regularization term such as λ×||PW||^2 can be added to the objective function, where P is the projection matrix, W is the spatial correlation weight, and λ is the regularization coefficient. The preset independence threshold is set as needed and is not specifically limited. When the mutual information between dimensions is lower than the preset independence threshold, it indicates that these dimensions are sufficiently independent, and the iteration can be stopped. For example, a small positive number, such as 0.01 or 0.001, can be set as the threshold for mutual information.

[0064] Among them, the spatial morphology dimension represents the geometric structure, physical layout and visual characteristics of the block, such as building height, density, street width, facade texture, etc.; the functional composition dimension represents the types of activities, business formats and functional mixing within the block, such as the proportion and spatial distribution of different functions such as commerce, residence, office and leisure; the spatial location dimension represents the relative position, accessibility and connection with other areas of the block in a larger geographical area, such as the distance from the city center and transportation hubs, and its level in the urban network.

[0065] S107. The dynamic evolution trend information is used as the second type of attribute information, and the second type of attribute information and the first type of attribute information are integrated to obtain the dynamic basic representation information of the target block.

[0066] Specifically, the potential transition direction probability vector in the second type of attribute information is used as the time driving factor, and the spatial location dimension in the first type of attribute information is used as the spatial carrying capacity basis. Based on the conditional coding layer of the conditional variational autoencoder, the potential transition direction probability vector and the expected transition time window are jointly probabilistically encoded using the spatial carrying capacity basis as the conditional variable to generate a spatiotemporal coupled latent vector. The spatiotemporal coupled latent vector, the spatial morphology dimension and the functional composition dimension are concatenated to obtain the concatenated features. The concatenated features are transformed into dynamic basic representation information by the decoder. The dynamic basic representation information is a four-dimensional tensor structure. The first and second dimensions correspond to the horizontal and vertical coordinates of the target block spatial grid, respectively. The third dimension corresponds to the prediction time window, and the fourth dimension corresponds to the attribute transition probability distribution of each spatial grid within the corresponding prediction time window.

[0067] In the second type of attribute information, the potential transition direction probability vector represents the potential tendency of the target block to undergo attribute transitions in the future, i.e., the probability distribution of transitions from the current state to other possible states. Using it as a time-driving factor indicates that the potential transition direction probability vector plays a role in guiding or triggering the temporal evolution process in the model. Its numerical changes or distribution patterns directly affect the generation of subsequent spatiotemporally coupled latent vectors, thereby driving the prediction of future dynamic changes in the block. For example, the potential transition direction probability vector can be a multidimensional probability distribution, where each dimension corresponds to a potential semantic category, such as commercial, residential, or industrial, and its value represents the probability of transitioning to that category.

[0068] The spatial location dimension in the first type of attribute information is used to characterize the location and geometric layout of the target block in geospace. Using it as a spatial basis indicates that it forms the foundation for mapping and locating all subsequent dynamic evolution information in space. This spatial basis provides a clear geospatial context for the evolution of dynamic attributes, ensuring that the prediction results can be associated with specific spatial locations. For example, the spatial location dimension can consist of a set of geographic coordinate points or a gridded spatial index, where each point or grid cell represents a specific area within the block. Alternatively, it can be a local coordinate system defined based on the relative positions of landmarks or key structures within the block.

[0069] The Conditional Variational Autoencoder (CVAE) is a deep generative model that adds conditional variables to the traditional variational autoencoder, enabling the model to generate or encode data based on given conditions. Its conditional encoding layer is a core component of CVAE, responsible for jointly encoding input data, such as potential transition direction probability vectors and expected transition time windows, with conditional variables, such as spatial carrying capacity, into a low-dimensional latent vector. This conditional encoding layer consists of multiple neural networks, such as convolutional neural networks or recurrent neural networks, to process the input data and fuse the conditional variables with intermediate features through fully connected layers, ultimately outputting the mean and variance of the latent vector. In other optional implementations, the conditional encoding layer can also employ an attention mechanism, allowing the model to adaptively focus on the spatially relevant parts of the input data during the encoding process, guided by the conditional variables.

[0070] In this embodiment, by inputting the spatial carrying capacity basis as a conditional variable into the conditional encoding layer, the model learns how to associate dynamic trends with specific spatial locations when generating latent vectors. Joint probabilistic encoding indicates that the model not only encodes the deterministic features of this information, but also its uncertainty or variability, thereby generating a latent vector that can capture multiple possible evolution paths. For example, joint probabilistic encoding can be used as the input of the CVAE encoder by concatenating the potential transition direction probability vector and the expected transition time window, while introducing the spatial carrying capacity basis as an additional input layer or through a gating mechanism.

[0071] The spatiotemporally coupled latent vector (SPLC) is a low-dimensional vector representation obtained after processing by a conditional coding layer. It contains the potential direction and temporal information of future attribute transitions of the target street at a specific spatial location. The spatiotemporal coupling characteristic of the SPLC is reflected in the fact that each dimension implicitly encodes spatial and temporal correlations, enabling subsequent decoding to generate representations with spatial positioning and temporal dynamics. For example, the SPLC can be a fixed-length floating-point vector whose internal structure is learned through a neural network, effectively compressing and abstracting the original spatiotemporal dynamic information. Furthermore, the SPLC can also be a hierarchical structure, where one dimension encodes spatial information and another encodes temporal information, but there are complex nonlinear relationships between them.

[0072] Feature concatenation involves connecting feature vectors from different sources or with different semantics along a dimensional plane to form a longer, more comprehensive feature vector containing richer information. For example, feature concatenation can be achieved through vector concatenation, where three vectors are joined end-to-end. Alternatively, these features can be subjected to independent nonlinear transformations, such as through a small fully connected network, before concatenation.

[0073] In this embodiment, the decoder is another core component of the conditional variational autoencoder, used to restore or map the low-dimensional latent vectors generated by the encoder back to the high-dimensional original data space, that is, to generate the final dynamic basic representation information. The decoder is composed of a multi-layer neural network, for example, using deconvolutional layers or upsampling layers to gradually restore the spatial dimension, and using fully connected layers or attention mechanisms to generate the attribute transition probability density distribution.

[0074] In this structure, the first and second dimensions represent the horizontal and vertical coordinates of the target block's spatial grid, respectively. The third dimension represents the prediction time window, and the fourth dimension stores the probability distribution of each spatial grid transitioning to different attribute categories within the corresponding prediction time window. This indicates that for each spatial point within the block, a probability prediction of its future functional or morphological changes can be obtained. For example, this four-dimensional tensor can be represented as P(Xd,Yd,Td,Cd), where Xd and Yd represent the spatial grid coordinates, Td represents the prediction time window, and Cd represents the attribute category.

[0075] In this embodiment, the potential transition direction probability vector is used as a time-driven factor, and the spatial location dimension is used as a spatial carrying basis. Based on the conditional coding layer of the conditional variational autoencoder, the spatial carrying basis is used as a conditional variable to jointly code the potential transition direction probability vector and the expected transition time window, thereby generating a spatiotemporally coupled latent vector. This transforms the discrete evolution trend into a continuous probability space expression, ensuring that dynamic information is closely related to the specific geographic spatial location.

[0076] After obtaining the dynamic basic representation information of the target block, the process further includes: marginalizing the attribute transition probability density distribution in the dynamic basic representation information along the time axis to obtain the marginal transition confidence of the target block at each spatial location; identifying spatial locations with marginal transition confidence less than a preset confidence threshold as uncertain regions and obtaining the corresponding original data segments of the uncertain regions in the aligned multi-source spatiotemporal data; re-inputting the corresponding original data segments into the step of feature merging of static attribute information, current-moment point of interest attribute data of aligned multi-source spatiotemporal data, and geospatial association data, and increasing the attention weight coefficient of the original data segments in cross-modal attention fusion to generate corrected fused feature information; updating the dynamic basic representation information based on the corrected fused feature information until the marginal transition confidence is greater than the preset confidence threshold or the preset iteration limit is reached.

[0077] In this embodiment, marginalization along the time axis involves integrating or summing the attribute transition probability density distribution over the time dimension. This accumulates or averages the probabilities over the time dimension to obtain the total or average probability of attribute transitions occurring at each spatial location, i.e., the marginal transition confidence. For example, for each spatial grid coordinate (x, y), the transition probabilities P(x, y, t, c) at each prediction time window t and attribute category c can be summed or weighted according to a preset rule to obtain the marginal transition confidence Pm(x, y).

[0078] The preset confidence threshold can be set according to actual application needs, expert experience, or through methods such as cross-validation during model training. For example, if it is set to 0.7, when the confidence of the marginal transition of a certain spatial location is lower than 70%, the prediction result of that region is deemed unreliable. By comparing the confidence of the marginal transition of each spatial location with the preset confidence threshold, spatial regions with low reliability in the model prediction results can be identified.

[0079] After identifying the uncertain regions, the corresponding original data segments of the uncertain regions in the aligned multi-source spatiotemporal data are obtained. Based on the spatial coordinates and time range of these regions, the original data segments corresponding to these uncertain regions in spatiotemporal space are extracted from the aligned multi-source spatiotemporal data through data indexing or spatial querying.

[0080] In this embodiment, graph convolution operations can be performed only on the extracted original data segments to generate local fusion feature information for these uncertain regions. At the same time, the attention weight coefficient of the original data segments is increased in cross-modal attention fusion to generate corrected fusion feature information. When performing cross-modal attention fusion, an attention weight coefficient higher than the normal value is assigned to the original data segments from uncertain regions. For example, by multiplying by a preset amplification factor, such as 1.2 or 1.5, or by dynamically adjusting the weight according to the degree of insufficient confidence.

[0081] In this embodiment, the newly generated corrected fused feature information is re-input into the subsequent decomposition and integration steps to recalculate and update the dynamic basic representation information. Then, marginalization is performed again to check whether the new marginal transition confidence meets the preset confidence threshold. The iterative process will continue until the marginal transition confidence of all spatial locations reaches or exceeds the preset confidence threshold, or reaches the preset maximum number of iterations, such as 5 or 10 times, to prevent infinite loops and ensure the convergence of the optimization process.

[0082] In this embodiment, the marginal transition confidence of each spatial location is extracted by the probability density distribution of marginal attribute transitions. Areas with low prediction reliability are located, and areas with insufficient confidence are regarded as uncertain areas. The data is then backtracked to the aligned original data fragments and re-inputted into the feature merging step. The weights are dynamically adjusted in the cross-modal attention fusion to compensate for the uncertainty at the feature level. The dynamic basic representation is iteratively updated until the confidence threshold is met or the upper limit is reached, thereby improving the accuracy of prediction.

[0083] Based on the urban block construction method integrating street view vision provided in the above embodiments, this application also provides a specific implementation of the urban block construction device integrating street view vision. Please refer to the following embodiments.

[0084] First see Figure 2 The urban block construction device 200 that integrates street view vision provided in this application embodiment includes: The acquisition module 201 is used to acquire multimodal data of the target street at multiple consecutive historical time points. The multimodal data includes street view panoramic image sequences, interest point semantic attribute sequences, and geographic proximity relationship data. Alignment module 202 is used to perform temporal alignment processing on multimodal data to obtain aligned multi-source spatiotemporal data; Extraction module 203 is used to extract static attribute information of the target block from the aligned multi-source spatiotemporal data, wherein the static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time. The generation module 204 is used to generate dynamic evolution trend information of the target block based on the change pattern of the interest point attribute data of the aligned multi-source spatiotemporal data on the time axis. The dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transition in the future. The merging module 205 is used to merge the static attribute information, the current point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial correlation data to obtain the fused feature information of the target block. The decomposition module 206 is used to decompose the fused feature information into a first type of attribute information based on a preset attribute independence constraint. The first type of attribute information includes at least a spatial morphology dimension, a functional composition dimension, and a spatial location dimension. The integration module 207 is used to use dynamic evolution trend information as the second type of attribute information, and to integrate the second type of attribute information with the first type of attribute information to obtain the dynamic basic representation information of the target block.

[0085] As an optional implementation in this embodiment, the alignment module 202 is specifically used for: The street view panoramic image sequence, the point of interest semantic attribute sequence, and the geographic proximity relation data are mapped to a unified spatiotemporal coordinate system to obtain initial spatiotemporal aligned data. Sliding window sampling is performed on the initial spatiotemporal aligned data with a preset time window step size to extract the image texture feature tensor, semantic tag frequency vector, and spatial adjacency matrix within each time window. The structural similarity index of the image texture feature tensor of adjacent time windows is calculated, and scene abrupt frames are determined based on the fluctuation amplitude of the structural similarity index along the time axis. A reference time axis is constructed based on the timestamps of the scene abrupt frames. The semantic tag frequency vector is projected onto the reference time axis through nearest neighbor matching to obtain corrected point of interest temporal data corresponding to the frame level of the street view panoramic image sequence. The spatial adjacency matrix is ​​linearly interpolated according to the time interval of the reference time axis to obtain a spatial adjacency matrix synchronized with the corrected point of interest temporal data, thus forming aligned multi-source spatiotemporal data.

[0086] As an optional implementation in this embodiment, the generation module 204 is specifically used for: The semantic attribute sequence of interest points in the aligned multi-source spatiotemporal data is expanded along the time axis to obtain the temporal evolution curve of the proportion of interest point categories. The static attribute information is used as a spatial constraint mask to perform spatial weighting on the temporal evolution curve to obtain the denoised evolution curve. The attribute dwell time period and attribute transition critical point are extracted from the denoised evolution curve, and the category entropy stability index and the gradient change magnitude of the attribute transition critical point within the preset range are calculated. A Markov state transition probability matrix is ​​constructed based on the category entropy stability index and the gradient change magnitude. The state direction corresponding to the probability greater than the preset transition threshold in the Markov state transition probability matrix is ​​used as the dynamic evolution trend information, which includes the potential transition direction probability vector and the corresponding expected transition time window.

[0087] As an optional implementation in this embodiment, the merging module 205 is specifically used for: A graph attention network is constructed using road topology skeleton data from static attribute information as a spatial guidance graph. The current-moment interest point attribute data is used as node features, and geospatial association data is used as edge weights. Graph convolution is then performed to obtain the interest point distribution features under spatial constraints. The street view facade outline baseline from static attribute information is used as the discriminator constraint of the generative adversarial network (GAN) to perform modal consistency verification on the interest point distribution features under spatial constraints, eliminating abnormal interest point feature components that do not conform to the geometric constraints of the street view facade outline baseline. The verified interest point distribution features and the current frame visual features of the street view panoramic image sequence are then fused through cross-modal attention to obtain fused feature information. Each feature dimension in the fused feature information carries spatial association weights with the static attribute information.

[0088] As an optional implementation in this embodiment, the decomposition module 206 is specifically used for: The spatial association weights associated with static attribute information in the fused feature information are obtained and used as the initial basis vectors for orthogonal projection. An orthogonal decomposition objective function is constructed in Hilbert space based on a preset attribute independence constraint. The orthogonal decomposition objective function is used to minimize the mutual information between the spatial morphology dimension, functional composition dimension, and spatial location dimension. The fused feature information is substituted into the orthogonal decomposition objective function for iterative projection. In each iteration, the spatial association weights are used as a regularization term to constrain the projection direction until the mutual information between each dimension is lower than a preset independence threshold. The three orthogonal components obtained by projection at the time of stopping iteration are used as the spatial morphology dimension, functional composition dimension, and spatial location dimension, respectively.

[0089] As an optional implementation in this embodiment, the integration module 207 is specifically used for: The potential transition direction probability vector in the second type of attribute information is used as the time driving factor, and the spatial location dimension in the first type of attribute information is used as the spatial carrying capacity basis. Based on the conditional coding layer of the conditional variational autoencoder, the potential transition direction probability vector and the expected transition time window are jointly probabilistically encoded using the spatial carrying capacity basis as the conditional variable to generate a spatiotemporally coupled latent vector. The spatiotemporally coupled latent vector, the spatial morphology dimension and the functional composition dimension are concatenated to obtain the concatenated features. The concatenated features are transformed into dynamic basic representation information by the decoder. The dynamic basic representation information is a four-dimensional tensor structure. The first and second dimensions correspond to the horizontal and vertical coordinates of the target block spatial grid, respectively. The third dimension corresponds to the prediction time window, and the fourth dimension corresponds to the attribute transition probability distribution of each spatial grid within the corresponding prediction time window.

[0090] As an optional implementation in this embodiment, the extraction module 203 is specifically used for: Principal component analysis (PCA) is performed on the image texture feature tensors in the aligned multi-source spatiotemporal data along the time axis to obtain the eigenvalues ​​and eigenvectors corresponding to each principal component. The image texture feature vectors at each historical time node are projected onto the eigenvectors corresponding to each principal component to obtain the projection value sequence of each principal component. The temporal variance or low-frequency energy proportion of the projection value sequence of each principal component along the time axis is calculated, and the principal components with a temporal variance lower than a preset stability threshold or a low-frequency energy proportion higher than a preset proportion are identified as temporally stable principal components. The selected eigenvectors are then reconstructed into a static texture base. Edge detection and line fitting are performed based on the static texture base to extract the street scene facade outline baseline. Semantic segmentation and road centerline refinement are performed based on the static texture base to extract road topology skeleton data. The road topology skeleton data is used to provide spatial location anchors in the feature merging step of static attribute information, current point of interest attribute data of the aligned multi-source spatiotemporal data, and geospatial correlation data.

[0091] Figure 3 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0092] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0093] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0094] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 302 may include removable or non-removable (or fixed) media, or memory 302 may be a non-volatile solid-state memory. Memory 302 may be internal or external to an electronic device.

[0095] In one instance, memory 302 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0096] Memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform operations described with reference to a method for constructing urban blocks by fusing street view vision according to a first aspect of this disclosure.

[0097] The processor 301 reads and executes computer program instructions stored in the memory 302 to achieve... Figure 1 The embodiment shown illustrates a method for constructing urban blocks by integrating street view visuals.

[0098] In one example, the electronic device may also include a communication interface 303 and a bus 304. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 304 and complete communication with each other.

[0099] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0100] Bus 304 includes hardware, software, or both, that couples components of an electronic device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 304 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0101] This electronic device can execute a method for constructing urban blocks by integrating street view vision, as described in this application embodiment, thereby achieving a combination of... Figures 1-2 This describes a method and apparatus for constructing urban blocks that integrates street view visuals.

[0102] Furthermore, in conjunction with the urban block construction method integrating street view vision in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the urban block construction methods integrating street view vision in the above embodiments.

[0103] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0104] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0105] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0106] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0107] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for constructing urban blocks that integrates street view visuals, characterized in that, include: Acquire multimodal data of the target street at multiple consecutive historical time points, wherein the multimodal data includes street view panoramic image sequences, point of interest semantic attribute sequences, and geographic proximity relationship data; The multimodal data is subjected to time-series alignment processing to obtain aligned multi-source spatiotemporal data; Static attribute information of the target block is extracted from the aligned multi-source spatiotemporal data, wherein the static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time. Based on the changes in the static attribute information and the interest point attribute data of the aligned multi-source spatiotemporal data on the time axis, dynamic evolution trend information of the target block is generated, wherein the dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transitions in the future. The static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data are feature merged to obtain the fused feature information of the target block. Based on the preset attribute independence constraint, the fused feature information is decomposed into a first type of attribute information, wherein the first type of attribute information includes at least the spatial morphology dimension, the functional composition dimension, and the spatial location dimension. The dynamic evolution trend information is used as the second type of attribute information, and the second type of attribute information and the first type of attribute information are integrated to obtain the dynamic basic representation information of the target block.

2. The method for constructing urban blocks by integrating street view visuals according to claim 1, characterized in that, The time-series alignment process for the multimodal data includes: The street view panoramic image sequence, the point of interest semantic attribute sequence, and the geographic proximity relationship data are respectively mapped to a unified spatiotemporal coordinate system to obtain initial spatiotemporal alignment data; The initial spatiotemporal alignment data is sampled using a sliding window with a preset time window as the step size, and the image texture feature tensor, semantic label frequency vector and spatial adjacency matrix are extracted within each time window; Calculate the structural similarity index of the image texture feature tensor for adjacent time windows, and determine the scene mutation frame based on the fluctuation amplitude of the structural similarity index along the time axis; A baseline timeline is constructed based on the timestamps of the scene mutation frames. The semantic tag frequency vector is projected onto the reference time axis through nearest neighbor matching to obtain the corrected interest point time series data corresponding to the frame level of the street view panoramic image sequence. The spatial adjacency matrix is ​​linearly interpolated according to the time interval of the reference time axis to obtain a spatial adjacency matrix synchronized with the corrected interest point time series data, so as to form the aligned multi-source spatiotemporal data.

3. The method for constructing urban blocks by integrating street view visuals according to claim 1, characterized in that, The process of generating dynamic evolution trend information of the target street based on the changing patterns of the interest point attribute data on the time axis, using the static attribute information and the aligned multi-source spatiotemporal data, includes: Expanding the sequence of semantic attributes of interest points in the aligned multi-source spatiotemporal data along the time axis yields the temporal evolution curve of the proportion of interest point categories. The static attribute information is used as a spatial constraint mask to perform spatial weighting on the temporal evolution curve to obtain the denoised evolution curve. Extract the attribute dwell time period and attribute transition critical point from the denoised evolution curve, and calculate the category entropy stability index within the attribute dwell time period and the gradient change magnitude of the attribute transition critical point within a preset range. Construct a Markov state transition probability matrix based on the category entropy stability index and the gradient change magnitude; The state pointers corresponding to probabilities greater than a preset transition threshold in the Markov state transition probability matrix are used as the dynamic evolution trend information, wherein the dynamic evolution trend information includes a potential transition direction probability vector and a corresponding expected transition time window.

4. The method for constructing urban blocks by integrating street view visuals according to claim 1, characterized in that, The step of feature merging of the static attribute information, the current time point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial correlation data includes: Using the road topology skeleton data in the static attribute information as a spatial guidance graph, a graph attention network is constructed. The current time-based interest point attribute data is used as node features, and the geospatial association data is used as edge weights. Graph convolution operation is performed to obtain the interest point distribution features under spatial constraints. The street view facade outline baseline in the static attribute information is used as the discriminator constraint condition of the generative adversarial network. Modal consistency verification is performed on the distribution features of interest points under the spatial constraints, and abnormal interest point feature components that do not conform to the geometric constraints of the street view facade outline baseline are eliminated. The interest point distribution features that have passed the verification and the visual features of the current frame of the street view panoramic image sequence are fused together by cross-modal attention to obtain the fused feature information, wherein each feature dimension in the fused feature information carries a spatial association weight with the static attribute information.

5. The method for constructing urban blocks by integrating street view visuals according to claim 1, characterized in that, The process of decomposing the fused feature information into a first type of attribute information based on a preset attribute independence constraint includes: Obtain the spatial association weights associated with the static attribute information in the fused feature information, and use the spatial association weights as the initial basis vectors for orthogonal projection; Based on the preset attribute independence constraint, an orthogonal decomposition objective function is constructed in Hilbert space, wherein the orthogonal decomposition objective function is used to minimize the mutual information between the spatial morphology dimension, functional composition dimension and spatial location dimension; The fused feature information is substituted into the orthogonal decomposition objective function for iterative projection. In each iteration, the spatial correlation weight is used as a regularization term to constrain the projection direction until the mutual information between each dimension is lower than the preset independence threshold and the iteration stops. The three orthogonal components obtained by projection at the point of stopping iteration are respectively used as the spatial morphology dimension, functional composition dimension, and spatial location dimension.

6. The method for constructing urban blocks by integrating street view visuals according to claim 3, characterized in that, The integration of the second type of attribute information and the first type of attribute information includes: The potential transition direction probability vector in the second type of attribute information is used as the time driving factor, and the spatial location dimension in the first type of attribute information is used as the spatial carrying basis. Based on the conditional variational autoencoder, the conditional coding layer uses the spatial bearing substrate as the conditional variable to perform joint probabilistic coding on the potential transition direction probability vector and the expected transition time window to generate a spatiotemporally coupled latent vector. The spatiotemporal coupling latent vector, the spatial morphology dimension, and the functional composition dimension are concatenated to obtain the concatenated features. The splicing features are transformed into dynamic basic representation information by the decoder. The dynamic basic representation information is a four-dimensional tensor structure. The first and second dimensions correspond to the horizontal and vertical coordinates of the target block spatial grid, respectively. The third dimension corresponds to the prediction time window, and the fourth dimension corresponds to the attribute transition probability distribution of each spatial grid within the corresponding prediction time window.

7. The method for constructing urban blocks by integrating street view visuals according to claim 1, characterized in that, The extraction of static attribute information of the target block includes: Principal component analysis is performed on the image texture feature tensor in the aligned multi-source spatiotemporal data along the time axis to obtain the eigenvalues ​​and eigenvectors corresponding to each principal component. Project the image texture feature vectors of each historical time point onto the feature vectors corresponding to each principal component to obtain the sequence of projection values ​​of each principal component. Calculate the temporal variance or low-frequency energy ratio of the projection value sequence of each principal component along the time axis, and determine the principal components whose temporal variance is lower than a preset stability threshold or whose low-frequency energy ratio is higher than a preset ratio as temporally stable principal components, and reconstruct the feature vectors corresponding to the temporally stable principal components into a static texture base. Edge detection and line fitting are performed based on the static texture base to extract the street scene facade outline baseline; Semantic segmentation and road centerline refinement are performed based on the static texture base to extract the road topology skeleton data; The road topology skeleton data is used to provide spatial location anchors in the feature merging step of the static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data.

8. A device for constructing urban blocks that integrates street view visuals, characterized in that, include: The acquisition module is used to acquire multimodal data of the target street at multiple consecutive historical time points, wherein the multimodal data includes street view panoramic image sequences, interest point semantic attribute sequences, and geographic proximity relationship data; The alignment module is used to perform temporal alignment processing on the multimodal data to obtain aligned multi-source spatiotemporal data; An extraction module is used to extract static attribute information of the target block from the aligned multi-source spatiotemporal data, wherein the static attribute information is used to characterize the inherent attributes of the target block in terms of spatial structure that do not change with time. The generation module is used to generate dynamic evolution trend information of the target block based on the change pattern of the static attribute information and the interest point attribute data of the aligned multi-source spatiotemporal data on the time axis. The dynamic evolution trend information is used to characterize the potential tendency of the target block to undergo attribute transition in the future. The merging module is used to merge the static attribute information, the current moment point of interest attribute data of the aligned multi-source spatiotemporal data, and the geospatial association data to obtain the fused feature information of the target block. The decomposition module is used to decompose the fused feature information into a first type of attribute information based on a preset attribute independence constraint, wherein the first type of attribute information includes at least a spatial morphology dimension, a functional composition dimension, and a spatial location dimension. The integration module is used to take the dynamic evolution trend information as the second type of attribute information, and integrate the second type of attribute information and the first type of attribute information to obtain the dynamic basic representation information of the target block.

9. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the urban block construction method that integrates street view vision as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the urban block construction method fused with street view vision as described in any one of claims 1-7.

Images