Railway real estate urban renewal method based on reinforcement learning

Abstract

This invention discloses a reinforcement learning-based method for urban renewal and transformation of railway real estate, comprising the following steps: collecting urban renewal and transformation scenario data of railway real estate and its surrounding urban space, implementing spatial unification and attribute standardization; constructing a coupled potential energy field state representation; constructing a set of transformation intervention action types, and constraining each transformation intervention action; constructing a sequential decision-making environment for urban renewal and transformation of railway real estate; constructing an improved MuZero model, generating a stage-aware state encoding vector in the sequential decision-making environment, combining it with the transformation intervention action, and performing potential energy decoding; generating a judgment identifier, determining the execution action based on the judgment identifier, and generating a continuous transformation path. This invention employs a reinforcement learning method that introduces potential energy constraints to perform multi-stage decision modeling of the urban renewal process of railway real estate, enabling the generation of continuously reachable transformation paths, and possessing the advantages of strong systematicity, high constraint consistency, and good implementation reliability.

Description

Technical Field

[0001] This invention relates to the field of intelligent planning technology, and in particular to a method for urban renewal and transformation of railway real estate based on reinforcement learning. Background Technology

[0002] Current urban renewal and renovation work on railway real estate mainly relies on manual planning experience, rule constraint analysis, and multi-indicator evaluation methods to comprehensively judge the functional configuration, development intensity, and connectivity of railway real estate land, line facilities, and surrounding urban spaces. Some technologies have begun to introduce GIS analysis, simulation and deduction, or machine learning-based auxiliary evaluation models to compare and select urban renewal schemes. However, the relevant methods are usually based on static indicator calculation or single-stage optimization, which makes it difficult to depict the evolutionary characteristics of multi-stage progress in the process of railway real estate renewal, and also makes it difficult to uniformly express the dynamic relationship between railway operation safety constraints and urban spatial structure.

[0003] Meanwhile, existing applications of reinforcement learning or planning algorithms in urban renewal often simplify the urban state to feature vectors or graph structures, lacking a holistic model of the interaction between railway real estate and the urban system. This makes it difficult to guarantee the continuous accessibility and consistency of the transformation process across multiple stages. Existing methods typically rely on immediate rewards or local feasibility judgments, which can easily lead to transformation paths that are feasible in one stage but unsustainable in subsequent stages. They cannot simultaneously meet the requirements of railway operation safety constraints, spatial evolution continuity, and stage target control during the transformation decision-making process, thus restricting the systematicness and reliability of urban renewal and transformation of railway real estate. Summary of the Invention

[0004] One objective of this invention is to propose a reinforcement learning-based method for urban renewal and transformation of railway real estate. This invention employs a reinforcement learning method that introduces potential energy constraints to perform multi-stage decision modeling of the urban renewal process of railway real estate, which can generate continuously reachable transformation paths and has the advantages of strong systematicity, high constraint consistency and good implementation reliability.

[0005] A method for urban renewal and renovation of railway real estate based on reinforcement learning according to an embodiment of the present invention includes the following steps: Collect urban renewal and renovation scenario data of railway real estate and its surrounding urban space, perform spatial unification and attribute standardization processing, and generate standardized urban renewal scenario data of railway real estate; Based on standardized railway real estate urban renewal scenario data, a coupled potential field state representation is constructed to characterize the interaction between railway real estate and urban systems; Construct a set of transformation intervention action types, constrain each transformation intervention action, and generate a set of transformation intervention actions; Using the coupled potential field state representation as the reinforcement learning state space and the set of renovation intervention actions as the reinforcement learning action space, a sequential decision-making environment for urban renewal and renovation of railway real estate is constructed. An improved MuZero model is constructed. In a sequential decision-making environment, a stage-aware state encoding vector is generated based on the coupled potential energy field state representation. This vector is then combined with the modification intervention action to generate the next stage implicit state encoding vector. Potential energy decoding is then performed to generate the next stage coupled potential energy field state representation. Based on the state representation of the coupled potential energy field in the next stage, a decision identifier is generated, the action to be executed is determined based on the decision identifier, and a continuous modification path is generated.

[0006] Optionally, the generation of standardized railway real estate urban renewal scenario data specifically includes: Collect spatial data of railway real estate within the scope of urban renewal and renovation of railway real estate; Collect attribute data of railway real estate within the scope of urban renewal and renovation of railway real estate; Collect urban spatial structure data connected to railway real estate; Collect railway operation safety constraint data; Unified spatial processing is performed on railway real estate spatial data, railway real estate attribute data, urban spatial structure data, and railway operation safety constraint data; Spatial scale unification is performed on the spatially unified data, converting continuous spatial data into discrete spatial unit representations; Perform attribute standardization on data that has achieved spatial scale uniformity to generate data representation results with consistent attributes; To introduce renewal stage identifiers for the urban renewal and renovation process of railway real estate, and to mark the standardized data at each stage. After spatial unification, attribute standardization, and stage marking, various types of data are collected to generate standardized railway real estate urban renewal scenario data.

[0007] Optionally, the generation of the coupled potential field state representation specifically includes: Acquire standardized urban renewal scenario data of railway real estate, and divide the standardized urban renewal scenario data of railway real estate into spatial units according to a unified spatial scale to form a basic spatial unit set; Based on the set of basic spatial units, a railway real estate space occupancy status layer is constructed, which maps the railway real estate land boundary, line direction, and spatial distribution of stations and ancillary facilities to the corresponding spatial units, generating the potential energy distribution of railway real estate space occupancy. A spatial connectivity layer is constructed based on urban spatial structure data, and the road connectivity structure and the spatial boundaries of adjacent plots are mapped to the corresponding spatial units to generate connectivity potential energy distribution. A functional configuration status layer is constructed based on railway real estate attribute data. The existing functional utilization status and utilization intensity identifiers are mapped to the corresponding spatial units to generate functional configuration potential energy distribution. A safety constraint layer is constructed based on railway operation safety constraint data, and safety control distance, facility protection range and operation risk classification limit are mapped to corresponding spatial units to generate safety constraint potential energy distribution; Spatial alignment is performed on the potential energy distribution of railway real estate space occupancy, connectivity, functional configuration, and safety constraint to form a multi-potential energy distribution result aligned under the same set of spatial units. Normalize the aligned multipotential distribution results; The normalized multi-potential energy distribution results are coupled and spliced ​​together according to the spatial unit order to form a coupled potential energy field state representation that characterizes the interaction between railway real estate and urban system.

[0008] Optionally, the generation of the set of modification intervention actions specifically includes: Within the scope of urban renewal and renovation of railway real estate, a set of renovation intervention action types is constructed, and action type identifiers are assigned to each renovation intervention action type. Determine the corresponding action space for each type of modification intervention action, and establish a mapping relationship between the type of modification intervention action and the spatial units in the set of spatial units to form a set of action space; For each type of intervention action, a set of action parameters is constructed, and the range of parameter values ​​is defined for each parameter item, forming a parameterized representation of the intervention action; Based on railway operation safety constraint data, the parameterized representation of the modification intervention action is subjected to constraint screening processing to remove action parameter values ​​that do not meet the railway operation safety constraint conditions and update the action parameter set. The action type identifier, action action space set, and updated action parameter set are combined and encoded to generate a set of intervention actions.

[0009] Optionally, the set of intervention types includes functional configuration adjustment actions, development intensity adjustment actions, spatial connectivity adjustment actions, and public service configuration adjustment actions.

[0010] Optionally, the generation of the sequential decision-making environment specifically includes: Obtain the state representation of the coupled potential energy field, which is defined as the state space representation in the reinforcement learning sequential decision-making environment; Obtain the set of intervention actions and use it as the action space representation in a reinforcement learning sequential decision-making environment; Based on the coupled potential energy field state representation and the set of intervention actions, a state-action mapping structure is constructed. To enhance the learning sequential decision-making environment, a state transition interface is configured, and the corresponding state representation of the coupled potential field for the next stage is output. To strengthen the sequential decision-making environment configuration sequence advancement control mechanism, the execution order of transformation intervention actions is controlled to form a state-action-state sequence that is advanced in stages; The output will be a reinforcement learning sequential decision-making environment that completes the definition of state space, action space, configuration of state transition interface, and configuration of sequence advancement control mechanism.

[0011] Optionally, the sequential decision-making environment is a decision-making operation environment that uses the state representation of the coupled potential energy field as the state input, the set of modification intervention actions as the action input, and gradually generates the state representation of the coupled potential energy field of the next stage according to the order of the modification stages.

[0012] Optionally, the generation of the next-stage coupled potential field state representation specifically includes: An improved MuZero model is constructed, including a state encoding network, a state evolution network, a potential energy conservation constraint mapping module, and a cross-stage potential energy evolution reachability constraint module; Obtain the state representation of the coupled potential energy field corresponding to the current transformation stage, and organize it into a state input matrix according to the order of the spatial unit set; The state input matrix is ​​input into the state encoding network, feature embedding mapping is performed on the spatial unit dimension, and aggregate encoding is performed on all spatial units to generate the state encoding vector. Obtain the stage sequence index corresponding to the current transformation stage, map it to a stage encoding vector, and concatenate and fuse the stage encoding vector with the state encoding vector to generate a stage-aware state encoding vector; In the state evolution network, the stage-aware state encoding vector and the modification intervention action are used as joint inputs to perform implicit state transition calculations and generate the implicit state encoding vector for the next stage. The implicit state encoding vector of the next stage is decoded using a multilayer perceptron to generate updated values ​​of potential energy distribution for railway real estate space occupancy, connectivity, functional configuration, and safety constraint for each spatial unit, thus forming the state representation of the coupled potential energy field of the next stage.

[0013] Optionally, the generation of the continuous modification path specifically includes: Input the state representation of the next stage coupled potential energy field into the potential energy conservation constraint mapping module to generate the total potential energy value of the next stage coupled potential energy field, and compare it with the preset target coupled potential energy field state interval to generate a potential energy conservation constraint satisfaction flag or a potential energy conservation constraint mismatch flag. In the cross-stage potential energy evolution reachability constraint module, and under the premise that the potential energy conservation constraint satisfies the flag, the state evolution network is used to perform multi-step implicit evolution reasoning to obtain the predicted implicit state encoding vector sequence for the subsequent stages, and generate the corresponding predicted coupled potential energy field state representation sequence. The total potential energy value of the coupled potential energy field in each prediction stage is compared with the preset target coupled potential energy field state interval to generate the cross-stage potential energy evolution reachability flag or the cross-stage potential energy evolution unreachability flag. The corresponding modification intervention action will be determined as the action to be executed in the current modification stage only when the potential energy conservation constraint satisfaction mark and the cross-stage potential energy evolution reachability mark are both satisfied. The coupled potential energy field state corresponding to the action is taken as the new current state, and the next transformation stage is entered. The constraint judgment and transformation intervention action determination process is repeated to determine the action to be performed at each stage in sequence, thus forming a continuous transformation path for urban renewal and transformation of railway real estate.

[0014] Optionally, the training of the improved MuZero model specifically includes: Based on the continuous transformation path, the actual transformation process is driven. After the actual transformation is completed, the state representation of the resulting coupled potential energy field is obtained and input into the state coding network to generate an implicit state coding vector. The implicit state encoding vector is compared with the implicit state encoding vector predicted by the state evolution network, and the joint deviation between the two in the implicit state space is calculated. The improved MuZero model is obtained by updating the parameters of the state encoding network and the state evolution network based on the joint bias.

[0015] The beneficial effects of this invention are: This invention constructs a coupled potential energy field state representation, which maps the spatial occupancy status of railway real estate, urban spatial connectivity, functional configuration status, and railway operation safety constraints to the same spatial unit scale for modeling. This allows the interaction between railway real estate and the urban system to be characterized holistically in a structured and computable form. Based on this, the urban renewal and transformation process of railway real estate is transformed into a stage-constrained reinforcement learning sequential decision-making problem. This makes the transformation decision no longer dependent on static indicators or single scheme comparisons, but achieves overall coordination and dynamic control of the multi-stage transformation process through continuous modeling of the state evolution process, thereby improving the systematicness and consistency of railway real estate urban renewal and transformation decision-making.

[0016] Furthermore, this invention introduces a potential energy conservation constraint mapping module and a cross-stage potential energy evolution accessibility constraint module into the improved MuZero model. This explicitly embeds the allowable potential energy evolution boundaries and multi-stage continuous accessibility requirements for each stage of railway real estate urban renewal and transformation into the model's reasoning and decision-making process. By jointly constraining candidate transformation intervention actions at single-stage and cross-stage scales, it effectively avoids transformation paths that only meet the conditions in local stages but cannot be continuously promoted in subsequent stages. This mechanism ensures the feasibility and continuity of continuous transformation paths in terms of spatial, functional, and safety constraints, thereby improving the reliability of transformation path generation results in actual implementation.

[0017] Meanwhile, this invention restricts the selectable range of renovation intervention actions through stage sequence indexing, so that different renovation stages correspond to different decision spaces, thereby maintaining consistency with the actual promotion logic of railway real estate urban renewal and renovation. By continuously updating the state representation of the coupled potential energy field during the reinforcement learning decision-making process, and combining the actual renovation execution results to train and correct the improved MuZero model, the model can gradually learn the state evolution relationship that conforms to the law of railway real estate renewal and renovation. This invention can not only generate continuous renovation paths that meet multiple constraints, but also improve the stability, feasibility and stage coordination of renovation decisions in complex renovation scenarios, providing an intelligent decision-making method with engineering feasibility for railway real estate urban renewal and renovation. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a reinforcement learning-based urban renewal and renovation method for railway real estate proposed in this invention. Figure 2 This is a schematic diagram of the improved MuZero model, which is a reinforcement learning-based method for urban renewal and transformation of railway real estate proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-2 A reinforcement learning-based method for urban renewal and redevelopment of railway real estate includes the following steps: Collect urban renewal and renovation scenario data of railway real estate and its surrounding urban space, perform spatial unification and attribute standardization processing, and generate standardized urban renewal scenario data of railway real estate; Based on standardized railway real estate urban renewal scenario data, a coupled potential field state representation is constructed to characterize the interaction between railway real estate and urban systems; Construct a set of transformation intervention action types, constrain each transformation intervention action, and generate a set of transformation intervention actions; Using the coupled potential field state representation as the reinforcement learning state space and the set of renovation intervention actions as the reinforcement learning action space, a sequential decision-making environment for urban renewal and renovation of railway real estate is constructed. An improved MuZero model is constructed. In a sequential decision-making environment, a stage-aware state encoding vector is generated based on the coupled potential energy field state representation. This vector is then combined with the modification intervention action to generate the next stage implicit state encoding vector. Potential energy decoding is then performed to generate the next stage coupled potential energy field state representation. Based on the state representation of the coupled potential energy field in the next stage, a decision identifier is generated, the action to be executed is determined based on the decision identifier, and a continuous modification path is generated.

[0021] In this embodiment, the generation of standardized railway real estate urban renewal scenario data specifically includes: Collect spatial data of railway real estate within the scope of urban renewal and renovation of railway real estate; Railway real estate spatial data includes railway real estate land boundaries, railway line alignment, and the spatial distribution of stations and ancillary facilities; Collect railway real estate attribute data within the scope of urban renewal and renovation of railway real estate. The railway real estate attribute data includes real estate ownership type, existing function utilization status and utilization intensity identifier. Collect urban spatial structure data connected to railway real estate, including road connectivity structure and spatial boundaries of adjacent plots; Collect railway operation safety constraint data, which includes safety control distance, facility protection range, and operational risk classification restrictions; Unified spatial processing is performed on railway real estate spatial data, railway real estate attribute data, urban spatial structure data, and railway operation safety constraint data; Spatial scale unification is performed on the spatially unified data, converting continuous spatial data into discrete spatial unit representations; Perform attribute standardization on data that has achieved spatial scale uniformity to generate data representation results with consistent attributes; To introduce renewal stage identifiers for the urban renewal and renovation process of railway real estate, and to mark the standardized data at each stage. After spatial unification, attribute standardization, and stage marking, various types of data are collected to generate standardized railway real estate urban renewal scenario data.

[0022] In this embodiment, the generation of the coupled potential field state representation specifically includes: Acquire standardized urban renewal scenario data of railway real estate, and divide the standardized urban renewal scenario data of railway real estate into spatial units according to a unified spatial scale to form a basic spatial unit set; Based on the set of basic spatial units, a railway real estate space occupancy status layer is constructed, which maps the railway real estate land boundary, line direction, and spatial distribution of stations and ancillary facilities to the corresponding spatial units, generating the potential energy distribution of railway real estate space occupancy. The railway real estate space occupancy status layer refers to the space occupancy status data layer formed by mapping the spatial distribution of railway real estate land boundaries, railway line directions, and station yards and ancillary facilities to corresponding spatial units under a unified set of spatial units. The railway real estate space occupancy potential energy distribution refers to the spatial potential energy distribution formed by numerically assigning values ​​to the occupancy status of railway real estate land, railway lines, and station yards and ancillary facilities in each spatial unit under a unified set of spatial units. It is used to reflect the impact of railway real estate physical occupancy on spatial adjustability. A spatial connectivity layer is constructed based on urban spatial structure data, mapping the road connectivity structure and the spatial boundaries of adjacent plots to the corresponding spatial units, generating a connectivity potential distribution, and reflecting the connection status between railway real estate and urban space. Connectivity potential energy distribution refers to the spatial potential energy distribution formed by numerically mapping the connectivity status between railway real estate spatial units and surrounding urban spatial units based on road connectivity structure and spatial connection relationship between adjacent plots. It is used to characterize the impact of accessibility and traffic conditions between spatial units on the evolution of urban renewal and transformation. A functional configuration status layer is constructed based on railway real estate attribute data. The existing functional utilization status and utilization intensity identifiers are mapped to the corresponding spatial units to generate functional configuration potential energy distribution. The existing functional utilization status and utilization intensity identifiers are used to describe the corresponding functional types and usage density of railway real estate in each spatial unit before renovation and upgrading. The functional configuration status layer is a spatial distribution layer formed by mapping the existing functional utilization status and utilization intensity identifiers according to a unified spatial unit. It is used to characterize the functional configuration status of railway real estate and its associated urban space. The functional configuration potential energy distribution is the spatial potential energy distribution result formed after numerical processing of the functional configuration status of each spatial unit in the functional configuration status layer. It is used to characterize the degree of influence of functional configuration on the urban renewal evolution of railway real estate. A safety constraint layer is constructed based on railway operation safety constraint data, and safety control distance, facility protection range and operation risk classification limit are mapped to corresponding spatial units to generate safety constraint potential energy distribution; Safety constraint potential energy distribution refers to the spatial potential energy distribution formed after mapping the safety control distance, facility protection range and operational risk classification restrictions in railway operation safety constraint data to a unified spatial unit. It is used to characterize the limiting effect of railway operation safety constraints on the degree of modification of the spatial unit. Spatial alignment is performed on the potential energy distribution of railway real estate space occupancy, connectivity, functional configuration, and safety constraint to form a multi-potential energy distribution result aligned under the same set of spatial units. Normalize the aligned multipotential distribution results; The normalized multi-potential energy distribution results are coupled and spliced ​​together according to the spatial unit order to form a coupled potential energy field state representation that characterizes the interaction between railway real estate and urban system.

[0023] In this embodiment, the generation of the set of modification intervention actions specifically includes: Within the scope of urban renewal and renovation of railway real estate, a set of renovation intervention action types is constructed, and action type identifiers are assigned to each renovation intervention action type. Determine the corresponding action space for each type of modification intervention action, and establish a mapping relationship between the type of modification intervention action and the spatial units in the set of spatial units to form a set of action space; The action action space set is a set of spatial indexes formed within the scope of urban renewal and renovation of railway real estate, after spatially defining the renovation intervention actions based on the spatial unit set. It is used to characterize the specific spatial unit range in which each renovation intervention action type can exert its effect. The action action space set is composed of spatial unit identifiers, each spatial unit identifier corresponds to a specific spatial unit, and each renovation intervention action type is associated with one or more spatial unit identifiers, limiting the renovation intervention action type to only take effect within the associated spatial units. For each type of intervention action, a set of action parameters is constructed, and the range of parameter values ​​is defined for each parameter item, forming a parameterized representation of the intervention action; The action parameter set is a set of parameters defined for each type of rehabilitation intervention action and their value ranges, used to describe the adjustable content of rehabilitation intervention actions at the numerical level. Based on railway operation safety constraint data, the parameterized representation of the modification intervention action is subjected to constraint screening processing to remove action parameter values ​​that do not meet the railway operation safety constraint conditions and update the action parameter set. The action type identifier, action action space set, and updated action parameter set are combined and encoded to generate a set of intervention actions.

[0024] In this embodiment, the set of transformation intervention actions includes functional configuration adjustment actions, development intensity adjustment actions, spatial connectivity adjustment actions, and public service configuration adjustment actions. Functional configuration adjustment actions refer to transformation intervention actions that change, replace, or redistribute the existing functional utilization status indicators within the railway real estate spatial unit and its associated urban spatial units, used to adjust the functional type structure carried by the spatial unit. Development intensity adjustment actions refer to transformation intervention actions that increase, decrease, or reset the development intensity parameters that characterize the degree of spatial development and utilization within the railway real estate spatial unit and its associated urban spatial units, used to control the construction scale and utilization density of the spatial unit. Spatial connectivity adjustment actions refer to transformation intervention actions that adjust the road connectivity structure, traffic relationship, or connection status indicators between the railway real estate spatial unit and surrounding urban spatial units, used to change the accessibility relationship and connectivity mode between spatial units. Public service configuration adjustment actions refer to transformation intervention actions that add, remove, or reconfigure the public service facility configuration status indicators within the railway real estate spatial unit and its associated urban spatial units, used to adjust the public service supply status corresponding to the spatial unit.

[0025] In this embodiment, the generation of the sequential decision-making environment specifically includes: Obtain the state representation of the coupled potential energy field, which is defined as the state space representation in the reinforcement learning sequential decision-making environment; Obtain the set of intervention actions and use it as the action space representation in a reinforcement learning sequential decision-making environment; Based on the state representation of the coupled potential energy field and the set of modification intervention actions, a state-action mapping structure is constructed to describe the modification intervention actions that can be selected under a given coupled potential energy field state. To enhance the learning sequential decision-making environment, a state transition interface is configured to receive the current coupled potential energy field state representation and the selected modification intervention action, and output the corresponding next-stage coupled potential energy field state representation. To strengthen the sequential decision-making environment configuration sequence advancement control mechanism, the execution order of transformation intervention actions is controlled to form a state-action-state sequence that is advanced in stages; The sequence advancement control mechanism includes: in a reinforcement learning sequential decision-making environment, setting a stage sequence index for the urban renewal and transformation process of railway real estate, limiting the selectable range of transformation intervention actions in different decision-making stages through the stage sequence index, allowing the selection of transformation intervention actions only from the set of railway real estate urban renewal and transformation intervention actions corresponding to the current stage sequence index in each decision-making stage, triggering the stage sequence index update after the transformation intervention action is selected and executed, advancing the current stage sequence index to the next stage sequence index, and driving the state representation of the coupled potential energy field to be updated from the current stage state to the next stage state, thereby forming a stage-constrained state-action-state sequence; The output will be a reinforcement learning sequential decision-making environment that completes the definition of state space, action space, configuration of state transition interface, and configuration of sequence advancement control mechanism.

[0026] In this embodiment, the sequential decision-making environment is a decision-making operation environment that uses the state representation of the coupled potential energy field as the state input, the set of modification intervention actions as the action input, and gradually generates the state representation of the coupled potential energy field of the next stage according to the order of the modification stages.

[0027] In this embodiment, the generation of the state representation of the coupling potential field in the next stage specifically includes: An improved MuZero model is constructed, including a state encoding network, a state evolution network, a potential energy conservation constraint mapping module, and a cross-stage potential energy evolution reachability constraint module; The state encoding network is used to encode the state representation of the coupled potential energy field. The state evolution network is used to perform joint mapping calculation on the stage-aware state encoding vector and the action encoding vector corresponding to the transformation intervention action under the stage sequence index constraint. It models the implicit state transition relationship after the transformation intervention action is applied to the state representation of the coupled potential energy field, and generates an implicit state encoding vector that represents the implicit evolution result of the state of the coupled potential energy field in the next stage. This enables the improved MuZero model to learn the implicit evolution relationship of the state of the coupled potential energy field that conforms to the phased advancement characteristics of the urban renewal and transformation process of railway real estate. The potential energy conservation constraint mapping module is used to perform overall potential energy consistency judgment on the state representation of the coupled potential energy field in the next stage, ensuring that the transformation evolution process conforms to the preset potential energy evolution boundary of each transformation stage. The cross-stage potential energy evolution reachability constraint module is used to verify the continuous reachability of the predicted evolution trajectory at multiple stage scales, avoiding transformation paths that are locally feasible but globally unreachable. The improved MuZero model introduces a potential energy conservation constraint mapping module and a cross-stage potential energy evolution reachability constraint module on the basis of the state encoding network and state evolution network. It explicitly embeds the stage target constraints and multi-stage evolution reachability constraints of the coupled potential energy field state in the process of urban renewal and transformation of railway real estate into the model inference process. This allows the model to learn the implicit evolution relationship of the coupled potential energy field state under the action of transformation intervention, while performing stage consistency judgment and cross-stage continuity verification on the predicted coupled potential energy field state representation of the next stage. This avoids transformation paths that are feasible only in a single stage but unsustainable at multiple stage scales. Through the above improvements, the improved MuZero model is transformed from a simple state evolution prediction model into a decision model that is simultaneously controlled by stage potential energy targets and cross-stage evolution constraints, realizing joint modeling of the feasibility, continuity and stage consistency of the urban renewal and transformation path of railway real estate. Obtain the state representation of the coupled potential energy field corresponding to the current transformation stage, and organize it into a state input matrix according to the order of the spatial unit set. The row index of the state input matrix corresponds to the spatial unit identifier, and the column vector is formed by splicing the normalized values ​​of the potential energy distribution of railway real estate space occupation, connectivity potential energy distribution, functional configuration potential energy distribution and safety constraint potential energy distribution under the same spatial unit. The state input matrix is ​​input into the state encoding network, feature embedding mapping is performed on the spatial unit dimension, and aggregate encoding is performed on all spatial units to generate the state encoding vector. The state encoding network uses a Graphormer network. Obtain the stage sequence index corresponding to the current transformation stage, map it to a stage encoding vector, and concatenate and fuse the stage encoding vector with the state encoding vector to generate a stage-aware state encoding vector, which is used to characterize the spatial potential energy distribution characteristics of the coupled potential energy field state under the current transformation stage. In the state evolution network, the stage-aware state encoding vector and the modification intervention action are used as joint inputs to perform implicit state transition calculations and generate the next stage implicit state encoding vector, which is used to characterize the implicit evolution result of the coupled potential energy field state that conforms to multiple potential energy constraints. The state evolution network adopts the dynamic network in the MuZero model. It takes the stage-aware state encoding vector and the modification intervention action as input, and performs nonlinear transformation on the state features through a multi-layer feedforward neural network to output the corresponding implicit state encoding vector of the next stage, which is used to characterize the implicit evolution result of the coupled potential field state under the action of modification intervention action. The implicit state encoding vector of the next stage is decoded using a multilayer perceptron to generate updated values ​​of potential energy distribution for railway real estate space occupancy, connectivity, functional configuration, and safety constraint for each spatial unit, thus forming the state representation of the coupled potential energy field of the next stage.

[0028] In this embodiment, the generation of continuous modification paths specifically includes: Input the state representation of the next stage coupled potential energy field into the potential energy conservation constraint mapping module to generate the total potential energy value of the next stage coupled potential energy field, and compare it with the preset target coupled potential energy field state interval to generate a potential energy conservation constraint satisfaction flag or a potential energy conservation constraint mismatch flag. When the total potential energy value of the next stage coupled potential energy field is within the preset target coupled potential energy field state range, it is an indicator that the potential energy conservation constraint is satisfied; when it is outside the range, it is an indicator that the potential energy conservation constraint is mismatched. When generating the total potential energy value of the next stage coupled potential energy field, the multi-type potential energy distribution of each spatial unit is converged to obtain the unit potential energy value, and the unit potential energy values ​​of all spatial units are accumulated to obtain the total potential energy value of the next stage coupled potential energy field. The preset target coupled potential energy field state range is the upper and lower boundary range of the total potential energy value of the coupled potential energy field in the next stage, which is determined in advance for each stage of urban renewal and renovation of railway real estate, based on the combined state of the potential energy distribution of railway real estate space occupation, connectivity, functional configuration, and safety constraint in the corresponding stage. In the cross-stage potential energy evolution reachability constraint module, and under the premise that the potential energy conservation constraint satisfies the flag, the state evolution network is used to perform multi-step implicit evolution reasoning to obtain the predicted implicit state encoding vector sequence for the subsequent stages, and generate the corresponding predicted coupled potential energy field state representation sequence. The total potential energy value of the coupled potential energy field in each prediction stage is compared with the preset target coupled potential energy field state interval to generate the cross-stage potential energy evolution reachability flag or the cross-stage potential energy evolution unreachability flag. The corresponding modification intervention action will be determined as the action to be executed in the current modification stage only when the potential energy conservation constraint satisfaction mark and the cross-stage potential energy evolution reachability mark are both satisfied. The state of the coupled potential energy field corresponding to the action is taken as the new current state, and the next transformation stage is entered. The process of constraint judgment and transformation intervention action determination is repeated, and the actions of each stage are determined in sequence to form a continuous transformation path of railway real estate urban renewal and transformation obtained by connecting the actions of multiple stages in sequence. When either the potential energy conservation constraint satisfaction flag or the cross-stage potential energy evolution reachability flag is not met, the corresponding modification intervention action is determined to be unexecutable under the current modification stage, and the modification intervention action is removed from the candidate execution action set of the current stage. Subsequently, the coupled potential energy field state representation corresponding to the current modification stage remains unchanged. For the remaining candidate modification intervention actions, implicit state evolution calculation, potential energy decoding generation, potential energy conservation constraint determination, and cross-stage potential energy evolution reachability determination are re-executed. When there is at least one modification intervention action that simultaneously satisfies the potential energy conservation constraint and the cross-stage potential energy evolution reachability constraint within the current modification stage, an execution action is selected from the modification intervention actions that satisfy the constraints and enters the actual modification execution process. When there is no modification intervention action that satisfies the above constraints within the current modification stage, the modification stage rollback control mechanism is triggered, the coupled potential energy field state representation of the current stage is maintained, and the modification intervention action selection process of the current stage is terminated, entering the next round of modification stage strategy generation and constraint evaluation process.

[0029] In this embodiment, the improved training of the MuZero model specifically includes: Based on the continuous transformation path, the actual transformation process is driven. After the actual transformation is completed, the state representation of the resulting coupled potential energy field is obtained and input into the state coding network to generate an implicit state coding vector. The implicit state encoding vector is compared with the implicit state encoding vector predicted by the state evolution network, and the joint deviation between the two in the implicit state space is calculated. The improved MuZero model is obtained by updating the parameters of the state encoding network and the state evolution network based on the joint bias. The parameter update is based on the joint bias. The network parameters of the state encoding network and the state evolution network are iteratively updated through backpropagation, so that the joint bias gradually converges during continuous training.

[0030] Example 1: To verify the feasibility and effectiveness of the present invention in practical applications, the present invention was applied to a scenario of collaborative renewal and transformation of existing railway real estate and urban built-up areas in a railway hub area and its surrounding areas. This area has long served the dual functions of railway transportation and urban traffic, and has problems such as large railway land scale, obvious spatial fragmentation, and insufficient utilization efficiency of surrounding urban functions. At the same time, it is subject to strict constraints such as railway operation safety control distance, facility protection range, and operational risks. The renewal and transformation process needs to gradually guide the reorganization of urban functions and optimization of spatial structure under the premise of ensuring the safe operation of railways. Traditional methods that rely on manual experience and static planning are difficult to continuously coordinate the multi-stage transformation process, and are prone to generating feasible solutions in some stages but difficult to continue to promote in subsequent stages.

[0031] In this application scenario, a unified model of the railway real estate and its surrounding urban space is first constructed. This involves collecting spatial distribution information on railway real estate land boundaries, line alignment, stations, and ancillary facilities. Simultaneously, the existing functional utilization status and intensity, as well as the connected urban road structure and adjacent land boundaries, are acquired. Furthermore, spatial unification and attribute standardization are performed in conjunction with railway operation safety-related control conditions. After processing, various data are organized into a set of discrete spatial units at a unified spatial scale, and update stage identifiers are introduced to reflect the state characteristics of railway real estate and urban space under different transformation stages. Based on this, potential energy distributions for railway real estate spatial occupancy, urban spatial connectivity, functional configuration, and safety constraints are constructed. These multiple potential energy types are aligned and coupled within the same set of spatial units to form a coupled potential energy field state representation characterizing the interaction between railway real estate and the urban system.

[0032] In practical applications, a reinforcement learning sequential decision-making environment is constructed based on the aforementioned coupled potential energy field state representation. A set of renovation intervention actions is constructed in conjunction with actual renovation needs. Actions such as functional configuration adjustment, development intensity adjustment, spatial connectivity adjustment, and public service configuration adjustment are introduced into the decision-making process in a constrained and parameterized form. The renovation process is divided by a stage sequence index, so that different stages correspond to different ranges of optional actions. This is consistent with the implementation logic of gradual and progressive advancement in the actual railway real estate renovation. In each renovation stage, the improved MuZero model generates a stage-aware state code based on the coupled potential energy field state representation of the current stage, and predicts the evolution result of the coupled potential energy field state in the next stage in conjunction with candidate renovation intervention actions.

[0033] In the process of transformation decision-making, potential energy conservation constraint mapping and cross-stage potential energy evolution reachability constraint are introduced to jointly determine each candidate transformation intervention action. By checking the constraint of the overall state of the coupled potential energy field in the next stage, the transformation behavior is prevented from exceeding the evolutionary boundary allowed by the stage. At the same time, the continuous reachability of the transformation path in subsequent stages is verified through multi-stage implicit evolutionary reasoning. Only transformation intervention actions that simultaneously meet the stage potential energy constraint and cross-stage reachability requirements are determined as the execution action of the current stage. Thus, the decision-making mechanism effectively avoids transformation schemes that are only reasonable in the current stage but difficult to sustain in subsequent stages.

[0034] In the actual application of this scenario, the transformation path is generated and implemented step by step in a phased sequence. After each phase is completed, the new coupled potential energy field state representation is fed back into the model for decision-making reasoning and model updates in subsequent phases. Through continuous iteration, the MuZero model is improved to gradually learn the evolution law of the coupled potential energy field state in the process of urban renewal and transformation of railway real estate, so that subsequent decisions are more in line with the actual spatial evolution and safety constraints.

[0035] From the perspective of application results, this invention successfully solves the problems of traditional methods in this scenario, namely, the difficulty in uniformly expressing the relationship between railway safety constraints and urban spatial evolution, and the difficulty in comprehensively coordinating multi-stage transformation processes. By coupling the potential energy field state representation with a reinforcement learning decision-making mechanism that introduces multiple constraints, the transformation process maintains good continuity between different stages, avoiding repeated adjustments to spatial functional configurations or frequent reconstruction of connectivity structures. Time records and on-site comparisons during the relevant implementation process show that the transformation path generated based on this invention has high stability during execution, significantly reduces the number of decision adjustments, and makes the transformation process smoother, effectively improving the controllability and reliability of urban renewal and transformation of railway real estate in actual implementation.

[0036] To verify the performance of the present invention in practice, a comparison was conducted, and the results are shown in Table 1.

[0037] Table 1. Comparison of the comprehensive performance of different urban renewal and redevelopment decision-making methods

[0038] As can be seen from Table 1, the traditional rule-driven planning method exhibits obvious stage instability characteristics in the process of urban renewal and transformation of railway real estate. The consistency rate of decision-making in the transformation stage is at a low level, while the rate of cross-stage scheme rollback and the number of recalculations of schemes in the middle are both high. This indicates that this type of method mainly relies on static rules and human experience. When facing a multi-stage continuous renewal scenario, it is difficult to effectively constrain the connection between the previous and subsequent stages, and it is easy to repeatedly adjust the previous scheme in the later stages.

[0039] Conventional reinforcement learning methods have improved the above problems to some extent. Their continuous reachability rate of transformation paths and consistency rate of stage decisions are both higher than those of traditional methods, indicating that the state-action-state learning mechanism can better adapt to the dynamic update process. However, data on the number of safety constraint violation adjustments and the occurrence rate of cross-stage solution rollback show that this type of method still has the problem of being mainly focused on single-stage optimization. Although it can obtain better decisions in local stages, the path will still be unsustainable at multi-stage scales, requiring additional adjustments to correct it.

[0040] In comparison, the method of this invention shows significant advantages in all indicators. The decision consistency rate and the continuous reachability rate of the transformation path are both at a high level, indicating that the decision-making mechanism based on the coupled potential energy field state representation and stage constraints can effectively maintain the decision-making continuity between different stages. The occurrence rate of cross-stage scheme rollback and the number of recalculations of mid-way schemes are reduced. This reflects that by jointly judging the potential energy conservation constraint and the cross-stage potential energy evolution reachability constraint, transformation paths that are difficult to continue in subsequent stages can be eliminated in advance during the decision-making stage, thereby reducing repeated adjustments during the implementation process.

[0041] Furthermore, the number of safety constraint violation adjustments decreased significantly, indicating that the present invention directly integrates railway operation safety constraints into the state modeling and action screening mechanism during the decision-making process. This makes safety constraints no longer dependent on ex-post verification, but rather become endogenous conditions for generating transformation paths. The reduction in decision convergence rounds reflects that the improved MuZero model, after introducing stage-aware encoding and multiple constraints, has a more stable learning of the transformation state evolution relationship, which helps to improve the overall decision-making efficiency.

[0042] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for urban renewal and redevelopment of railway real estate based on reinforcement learning, characterized in that, Includes the following steps: Collect urban renewal and renovation scenario data of railway real estate and its surrounding urban space, perform spatial unification and attribute standardization processing, and generate standardized urban renewal scenario data of railway real estate; Based on standardized railway real estate urban renewal scenario data, a coupled potential field state representation is constructed to characterize the interaction between railway real estate and urban systems; Construct a set of transformation intervention action types, constrain each transformation intervention action, and generate a set of transformation intervention actions; Using the coupled potential field state representation as the reinforcement learning state space and the set of renovation intervention actions as the reinforcement learning action space, a sequential decision-making environment for urban renewal and renovation of railway real estate is constructed. An improved MuZero model is constructed. In a sequential decision-making environment, a stage-aware state encoding vector is generated based on the coupled potential energy field state representation. This vector is then combined with the modification intervention action to generate the next stage implicit state encoding vector. Potential energy decoding is then performed to generate the next stage coupled potential energy field state representation. Based on the state representation of the coupled potential energy field in the next stage, a judgment identifier is generated, the action to be executed is determined based on the judgment identifier, and a continuous modification path is generated. The generation of the coupled potential field state representation specifically includes: Acquire standardized urban renewal scenario data of railway real estate, and divide the standardized urban renewal scenario data of railway real estate into spatial units according to a unified spatial scale to form a basic spatial unit set; Based on the set of basic spatial units, a railway real estate space occupancy status layer is constructed, which maps the railway real estate land boundary, line direction, and spatial distribution of stations and ancillary facilities to the corresponding spatial units, generating the potential energy distribution of railway real estate space occupancy. A spatial connectivity layer is constructed based on urban spatial structure data, and the road connectivity structure and the spatial boundaries of adjacent plots are mapped to the corresponding spatial units to generate connectivity potential energy distribution. A functional configuration status layer is constructed based on railway real estate attribute data. The existing functional utilization status and utilization intensity identifiers are mapped to the corresponding spatial units to generate functional configuration potential energy distribution. A safety constraint layer is constructed based on railway operation safety constraint data, and safety control distance, facility protection range and operation risk classification limit are mapped to corresponding spatial units to generate safety constraint potential energy distribution; Spatial alignment is performed on the potential energy distribution of railway real estate space occupancy, connectivity, functional configuration, and safety constraint to form a multi-potential energy distribution result aligned under the same set of spatial units. Normalize the aligned multipotential distribution results; The normalized multi-potential energy distribution results are coupled and spliced ​​together according to the spatial unit order to form a coupled potential energy field state representation that characterizes the interaction between railway real estate and urban system.

2. The method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 1, characterized in that, The generation of standardized railway real estate urban renewal scenario data specifically includes: Collect spatial data of railway real estate within the scope of urban renewal and renovation of railway real estate; Collect attribute data of railway real estate within the scope of urban renewal and renovation of railway real estate; Collect urban spatial structure data connected to railway real estate; Collect railway operation safety constraint data; Unified spatial processing is performed on railway real estate spatial data, railway real estate attribute data, urban spatial structure data, and railway operation safety constraint data; Spatial scale unification is performed on the spatially unified data, converting continuous spatial data into discrete spatial unit representations; Perform attribute standardization on data that has achieved spatial scale uniformity to generate data representation results with consistent attributes; To introduce renewal stage identifiers for the urban renewal and renovation process of railway real estate, and to mark the standardized data at each stage. After spatial unification, attribute standardization, and stage marking, various types of data are collected to generate standardized railway real estate urban renewal scenario data.

3. The method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 1, characterized in that, The generation of the set of intervention actions specifically includes: Within the scope of urban renewal and renovation of railway real estate, a set of renovation intervention action types is constructed, and action type identifiers are assigned to each renovation intervention action type. Determine the corresponding action space for each type of modification intervention action, and establish a mapping relationship between the type of modification intervention action and the spatial units in the set of spatial units to form a set of action space; For each type of intervention action, a set of action parameters is constructed, and the range of parameter values ​​is defined for each parameter item, forming a parameterized representation of the intervention action; Based on railway operation safety constraint data, the parameterized representation of the modification intervention action is subjected to constraint screening processing to remove action parameter values ​​that do not meet the railway operation safety constraint conditions and update the action parameter set. The action type identifier, action action space set, and updated action parameter set are combined and encoded to generate a set of intervention actions.

4. The method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 3, characterized in that, The set of intervention types includes functional configuration adjustment actions, development intensity adjustment actions, spatial connectivity adjustment actions, and public service configuration adjustment actions.

5. A method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 1, characterized in that, The generation of the sequential decision-making environment specifically includes: Obtain the state representation of the coupled potential energy field, which is defined as the state space representation in the reinforcement learning sequential decision-making environment; Obtain the set of intervention actions and use it as the action space representation in a reinforcement learning sequential decision-making environment; Based on the coupled potential energy field state representation and the set of intervention actions, a state-action mapping structure is constructed. To enhance the learning sequential decision-making environment, a state transition interface is configured, and the corresponding state representation of the coupled potential field for the next stage is output. To strengthen the sequential decision-making environment configuration sequence advancement control mechanism, the execution order of transformation intervention actions is controlled to form a state-action-state sequence that is advanced in stages; The output will be a reinforcement learning sequential decision-making environment that completes the definition of state space, action space, configuration of state transition interface, and configuration of sequence advancement control mechanism.

6. A method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 5, characterized in that, The sequential decision-making environment is a decision-making environment that uses the state representation of the coupled potential energy field as the state input, the set of modification intervention actions as the action input, and gradually generates the state representation of the coupled potential energy field for the next stage according to the order of the modification stages.

7. A method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 1, characterized in that, The generation of the next-stage coupled potential field state representation specifically includes: An improved MuZero model is constructed, including a state encoding network, a state evolution network, a potential energy conservation constraint mapping module, and a cross-stage potential energy evolution reachability constraint module; Obtain the state representation of the coupled potential energy field corresponding to the current transformation stage, and organize it into a state input matrix according to the order of the spatial unit set; The state input matrix is ​​input into the state encoding network, feature embedding mapping is performed on the spatial unit dimension, and aggregate encoding is performed on all spatial units to generate the state encoding vector. Obtain the stage sequence index corresponding to the current transformation stage, map it to a stage encoding vector, and concatenate and fuse the stage encoding vector with the state encoding vector to generate a stage-aware state encoding vector; In the state evolution network, the stage-aware state encoding vector and the modification intervention action are used as joint inputs to perform implicit state transition calculations and generate the implicit state encoding vector for the next stage. The implicit state encoding vector of the next stage is decoded using a multilayer perceptron to generate updated values ​​of potential energy distribution for railway real estate space occupancy, connectivity, functional configuration, and safety constraint for each spatial unit, thus forming the state representation of the coupled potential energy field of the next stage.

8. A method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 1, characterized in that, The generation of the continuous modification path specifically includes: Input the state representation of the next stage coupled potential energy field into the potential energy conservation constraint mapping module to generate the total potential energy value of the next stage coupled potential energy field, and compare it with the preset target coupled potential energy field state interval to generate a potential energy conservation constraint satisfaction flag or a potential energy conservation constraint mismatch flag. In the cross-stage potential energy evolution reachability constraint module, and under the premise that the potential energy conservation constraint satisfies the flag, the state evolution network is used to perform multi-step implicit evolution reasoning to obtain the predicted implicit state encoding vector sequence for the subsequent stages, and generate the corresponding predicted coupled potential energy field state representation sequence. The total potential energy value of the coupled potential energy field in each prediction stage is compared with the preset target coupled potential energy field state interval to generate the cross-stage potential energy evolution reachability flag or the cross-stage potential energy evolution unreachability flag. The corresponding modification intervention action will be determined as the action to be executed in the current modification stage only when the potential energy conservation constraint satisfaction mark and the cross-stage potential energy evolution reachability mark are both satisfied. The coupled potential energy field state corresponding to the action is taken as the new current state, and the next transformation stage is entered. The constraint judgment and transformation intervention action determination process is repeated to determine the action to be performed at each stage in sequence, thus forming a continuous transformation path for urban renewal and transformation of railway real estate.

9. A method for urban renewal and renovation of railway real estate based on reinforcement learning according to claim 7, characterized in that, The training of the improved MuZero model specifically includes: Based on the continuous transformation path, the actual transformation process is driven. After the actual transformation is completed, the state representation of the resulting coupled potential energy field is obtained and input into the state coding network to generate an implicit state coding vector. The implicit state encoding vector is compared with the implicit state encoding vector predicted by the state evolution network, and the joint deviation between the two in the implicit state space is calculated. The improved MuZero model is obtained by updating the parameters of the state encoding network and the state evolution network based on the joint bias.

Images