Energy project investment management method and system based on artificial intelligence

Abstract

The present invention provides an energy project investment management method and system based on artificial intelligence, which belongs to the field of artificial intelligence technology. First, energy project data is collected; secondly, satellite images of energy projects are obtained, and the satellite images are input into an image feature network model for feature extraction, and energy project environmental features are output; then the state space, action space and reward mechanism are defined; finally, a deep Q learning network is used to perform an energy investment decision-making process. The image feature network model structure of the present invention includes multiple convolutional layers, normalized activation layers, expanded receptive field modules, feature fusion modules, etc., which can effectively extract key information reflecting the environmental status of the project; in the specific implementation of reinforcement learning, the present invention constructs a deep Q network agent, uses experience replay and target Q network technology for iterative training and optimization, and the intelligent agent selects the optimal investment action according to its state in a simulated energy investment environment to maximize long-term cumulative rewards.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to an energy project investment management method and system based on artificial intelligence. Background Art

[0002] Against the backdrop of global energy transition and climate change, energy project investment decisions are becoming increasingly complex and challenging. This is not only because energy projects themselves involve comprehensive assessments of numerous technical, economic, and environmental factors, but also because the rapid evolution of energy markets, the uncertainty of the policy environment, and the increasing focus on sustainable development significantly increase investment risks and the difficulty of decision-making. To address these challenges, traditional investment decision-making methods based on experience and financial models are gradually showing their limitations. However, AI-based energy project investment management approaches offer a new solution. Summary of the Invention

[0003] Based on the above technical problems, the present invention provides an energy project investment management method and system based on artificial intelligence, which uses an image feature network model to extract features from satellite images of energy projects to obtain environmental characteristics, and uses a deep Q learning network to make intelligent investment decisions.

[0004] The present invention provides an energy project investment management method based on artificial intelligence, the method comprising:

[0005] Step S1: Collect energy project data;

[0006] Step S2: Obtain satellite images of energy projects, input the satellite images into an image feature network model for feature extraction, and output environmental features of the energy projects;

[0007] The image feature network model includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, a second normalized activation layer, an expanded receptive field module, a feature fusion module, a fourth upsampling layer, a second tensor splicing layer, a seventh standard convolution layer, a third normalized activation layer, an eighth standard convolution layer, a fourth normalized activation layer, a second global average pooling layer, a first fully connected layer and a second fully connected layer;

[0008] Step S3: Define the state space, action space and reward mechanism;

[0009] Step S4: Use deep Q-learning network for energy investment decision-making process.

[0010] Optionally, the collecting of energy project data specifically includes:

[0011] Collect data through API calls and web crawlers, set data crawling frequency, and implement data collection, including:

[0012] Import the libraries needed in Python. import requests is used to initiate HTTP requests; import json is used to process JSON data; import time provides various time-related functions; import datetime is used to process date and time;

[0013] Set the API URL; set the data type to JSON;

[0014] Design a data collection strategy. Set the Accept field in the HTTP request header to tell the server that the expected response data type is JSON. Use the User-Agent field to identify the client initiating the request, expressed as HEADERS. Define the frequency of data collection.

[0015] Define the function fetch_data_from_api to get data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS; if the response status code meets the requirements, the request is successful, and the .json() method is used to parse the response content into a Python dictionary in JSON format and return it; if the response status code does not meet the requirements, the request fails, and an error message is printed; and None is returned as the result;

[0016] Define the save_data_to_file function to save the acquired data to a local file; use the open function to open the file in write mode ('w'), and create it if it does not exist; use json.dump to write the data to the file in JSON format;

[0017] Define the main loop. The loop in the main function repeatedly performs data acquisition and saving operations at a set frequency. In each loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next cycle.

[0018] Optionally, the step of acquiring a satellite image of the energy project, inputting the satellite image into an image feature network model for feature extraction, and outputting environmental features of the energy project may specifically include:

[0019] Input the satellite image into the first standard convolution layer for convolution operation to obtain a feature map E1; input the feature map E1 into the first normalized activation layer for batch normalization and activation operation to obtain a feature map E2; input the feature map E2 into the second standard convolution layer for convolution operation to obtain a feature map E3; input the feature map E3 into the second normalized activation layer for batch normalization and activation operation to obtain the feature map E4;

[0020] Inputting the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16;

[0021] Input the feature map E16 into the feature fusion module to perform multi-feature fusion operation to obtain a feature map E31;

[0022] Input the feature map E31 to the fourth upsampling layer for upsampling operation to obtain a feature map E32; input the feature map E4 and the feature map E32 to the second tensor splicing layer for tensor splicing operation to obtain a feature map E33;

[0023] Inputting the feature map E33 into the seventh standard convolution layer for convolution operation to obtain a feature map E34; inputting the feature map E34 into the third normalized activation layer for batch normalization and activation operation to obtain a feature map E35;

[0024] Inputting the feature map E35 into the eighth standard convolution layer for convolution operation to obtain a feature map E36; inputting the feature map E36 into the fourth normalization activation layer for batch normalization and activation operation to obtain a feature map E37;

[0025] The feature map E37 is input into the second global average pooling layer for global average pooling operation to obtain a feature map E38; the feature map E38 is respectively input into the first fully connected layer and the second fully connected layer for full connection operation to output environmental features.

[0026] The present invention also provides an energy project investment management system based on artificial intelligence, the system comprising:

[0027] Energy project collection module, used to collect energy project data;

[0028] Satellite image acquisition module, used to obtain satellite images of energy projects, input the satellite images into the image feature network model for feature extraction, and output the environmental characteristics of the energy project;

[0029] The image feature network model includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, a second normalized activation layer, an expanded receptive field module, a feature fusion module, a fourth upsampling layer, a second tensor splicing layer, a seventh standard convolution layer, a third normalized activation layer, an eighth standard convolution layer, a fourth normalized activation layer, a second global average pooling layer, a first fully connected layer and a second fully connected layer;

[0030] Environment parameter setting module, used to define the state space, action space and reward mechanism;

[0031] Energy investment process module, which is used to conduct energy investment decision-making process using deep Q-learning network.

[0032] Optionally, the energy project collection module specifically includes:

[0033] Collect data through API calls and web crawlers, set data crawling frequency, and implement data collection, including:

[0034] Import the libraries needed in Python. import requests is used to initiate HTTP requests; import json is used to process JSON data; import time provides various time-related functions; import datetime is used to process date and time;

[0035] Set the API URL; set the data type to JSON;

[0036] Design a data collection strategy. Set the Accept field in the HTTP request header to tell the server that the expected response data type is JSON. Use the User-Agent field to identify the client initiating the request, expressed as HEADERS. Define the frequency of data collection.

[0037] Define the function fetch_data_from_api to get data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS; if the response status code meets the requirements, the request is successful, and the .json() method is used to parse the response content into a Python dictionary in JSON format and return it; if the response status code does not meet the requirements, the request fails, and an error message is printed; and None is returned as the result;

[0038] Define the save_data_to_file function to save the acquired data to a local file; use the open function to open the file in write mode ('w'), and create it if it does not exist; use json.dump to write the data to the file in JSON format;

[0039] Define the main loop. The loop in the main function repeatedly performs data acquisition and saving operations at a set frequency. In each loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next cycle.

[0040] Optionally, the satellite image acquisition module specifically includes:

[0041] The first two-standard convolution submodule inputs the satellite image into the first standard convolution layer for convolution operation to obtain a feature map E1; inputs the feature map E1 into the first normalized activation layer for batch normalization and activation operation to obtain a feature map E2; inputs the feature map E2 into the second standard convolution layer for convolution operation to obtain a feature map E3; inputs the feature map E3 into the second normalized activation layer for batch normalization and activation operation to obtain the feature map E4;

[0042] An expanded receptive field submodule, configured to input the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16;

[0043] A feature fusion submodule, configured to input the feature map E16 into the feature fusion module to perform a multi-feature fusion operation to obtain a feature map E31;

[0044] An upsampling submodule, configured to input the feature map E31 into the fourth upsampling layer for upsampling operation to obtain a feature map E32; input the feature map E4 and the feature map E32 into the second tensor splicing layer for tensor splicing operation to obtain a feature map E33;

[0045] a seventh standard convolution submodule, configured to input the feature map E33 into the seventh standard convolution layer for convolution operation to obtain a feature map E34; and input the feature map E34 into the third normalized activation layer for batch normalization and activation operation to obtain a feature map E35;

[0046] an eighth standard convolution submodule, configured to input the feature map E35 into the eighth standard convolution layer for convolution operation to obtain a feature map E36; and input the feature map E36 into the fourth normalization activation layer for batch normalization and activation operation to obtain a feature map E37;

[0047] The environmental feature output submodule is used to input the feature map E37 into the second global average pooling layer for global average pooling operation to obtain a feature map E38; input the feature map E38 into the first fully connected layer and the second fully connected layer for full connection operation, and output the environmental features.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention integrates big data collection, advanced image recognition technology and reinforcement learning algorithms, which improves the accuracy and automation level of energy project investment decisions and reduces decision-making bias caused by human factors; uses deep Q learning networks to optimize investment strategies, and intelligent agents can autonomously learn and adjust investment portfolios in complex market environments, enhancing the ability to cope with uncertainties such as market fluctuations and policy changes; by integrating satellite image feature extraction technology, the present invention incorporates geographic information into the decision-making process, making investment decisions more comprehensive and taking into account the impact of the actual environment on the benefits of energy projects; through simulation and training, it provides energy investors with an intelligent investment management system, which is conducive to improving investment performance and risk management levels, and has significant application value and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of the energy project investment management method based on artificial intelligence of the present invention;

[0051] Figure 2 This is a structural diagram of the image feature network model of the present invention;

[0052] Figure 3 This is a structural diagram of the feature fusion module of the present invention;

[0053] Figure 4 This is a structural diagram of the expanded receptive field module of the present invention;

[0054] Figure 5 This is a structural diagram of the energy project investment management system based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described below with reference to specific implementation cases and drawings, but the present invention is not limited to these embodiments.

[0056] Example 1

[0057] like Figure 1 As shown, the present invention discloses an energy project investment management method based on artificial intelligence, the method comprising:

[0058] Step S1: Collect energy project data.

[0059] Step S2: Obtain satellite images of energy projects, input the satellite images into the image feature network model for feature extraction, and output the environmental features of the energy projects.

[0060] Step S3: Define the state space, action space, and reward mechanism.

[0061] Step S4: Use deep Q-learning network for energy investment decision-making process.

[0062] The following describes each step in detail:

[0063] Step S1: Collect energy project data and perform preprocessing.

[0064] Step S1 specifically includes:

[0065] Identify data sources and data types, including:

[0066] Determine the data source. Before collecting data, you first need to clarify the type of data required. In the generation of energy investment projects, the required data mainly includes energy market data, policy documents, and project operation data. These data can help understand the dynamic changes in the market, the impact of policies, and the operation of projects, thereby providing strong support for investment decisions.

[0067] List specific data sources. Public data sources include government reports, energy industry white papers, statistical yearbooks, research reports released by research institutions, annual reports of listed companies, project information disclosed on official websites, etc.

[0068] Data type and content, basic attributes include basic information such as project name, geographical location (vegetation coverage, terrain), construction time, time to be put into operation, and expected retirement time; economic indicators include financial indicators such as construction cost, operating cost, expected annual income, total income during the expected life cycle, discount rate, etc.; technical characteristics include energy type (fossil fuel, wind power, solar energy, nuclear energy, etc.), production capacity, equipment specifications, technology maturity, maintenance frequency, etc.; environmental protection indicators include carbon emissions per unit of energy, pollutant emissions, proportion of renewable resource use, energy-saving and emission reduction measures, and other environmental protection performance indicators; policy orientation includes energy policies, subsidy policies, tax preferential policies, carbon trading market rules, and other factors affecting the economic benefits of the project at all levels of national and local governments.

[0069] Design a data collection strategy, including:

[0070] Choose an appropriate API calling method. For some data sources, directly obtain data through API calling. This method has the advantages of fast data acquisition speed and unified format. When choosing an API calling method, you need to understand the API calling rules, parameter settings, and return value format to ensure that you can correctly obtain the required data.

[0071] Web crawler technology can be used to collect data that cannot be obtained through API. A web crawler is an automated program that can simulate human behavior to capture and extract data on the Internet. When designing a web crawler, it is necessary to consider issues such as the crawler's stability, speed, and efficiency to ensure that the required data can be obtained efficiently.

[0072] Set the data crawling frequency. To ensure the real-time and integrity of the data, you need to set the data crawling frequency according to actual needs. For data that needs to be updated in real time, such as energy market prices, set a higher crawling frequency to obtain the latest data in a timely manner. For data that changes more slowly, such as policy documents, you can set a lower crawling frequency to reduce the pressure on the data source.

[0073] Implement data collection, including:

[0074] Write data collection scripts or programs. Before implementing data collection, write corresponding data collection scripts or programs according to the designed data collection strategy; these scripts or programs need to be able to automatically perform data capture tasks and store the captured data in a designated location; during the writing process, it is necessary to ensure the stability and accuracy of the scripts or programs to avoid data collection failures due to code errors or network problems.

[0075] Execute data collection tasks. After writing the data collection script or program, start executing the data collection task. During the execution process, pay close attention to the progress and status of data collection to ensure that the data collection task can proceed as expected. At the same time, the collected data needs to be preliminarily checked and cleaned to ensure the accuracy and completeness of the data.

[0076] Data storage and management: The collected data needs to be stored in a local or cloud database for subsequent data processing and analysis. During the storage process, issues such as the data format, naming conventions, and storage location need to be considered. At the same time, a data backup and recovery mechanism needs to be established to prevent data loss or damage. In addition, sensitive data or privacy data needs to be encrypted or desensitized to ensure data security.

[0077] In this example, we determined that real-time price data was obtained from an energy market API in JSON format. We used the Python requests library to call the API and set it to capture data once an hour. Specifically, we included the following:

[0078] Import the libraries needed in Python. import requests is used to initiate HTTP requests, which is used here to obtain data from the API; import json is used to process JSON data, such as converting Python dictionaries to JSON format or parsing data from JSON format; import time provides various time-related functions, which is used here to implement the waiting (pause execution) function; import datetime is used to process dates and times, such as generating timestamps.

[0079] Specify the data source and data type. Set the API URL as API_URL='https: / / example.com / api / energy_market_data'. Set the data type to JSON as DATA_TYPE='json'. This defines the data source (API URL) and data type (JSON). The API_URL here is an example URL and needs to be replaced with the actual API address in actual applications.

[0080] Design a data collection strategy, set the Accept field in the HTTP request header (Headers), tell the server that the expected response data type is JSON, and identify the client information initiating the request through User-Agent, expressed as HEADERS={'Accept':'application / json','User-Age nt':'MyEnergyProjectDataCollector / 1.0'}; define the frequency of data collection, here set to 3600 seconds, that is, once an hour, expressed as FREQUENCY=3600.

[0081] Implement data collection and define the function fetch_data_from_api to obtain data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS, expressed as response=requests.get(API_URL,headers=HEADERS); make a judgment, if the response status code is 200, it means the request is successful, use the .json() method to parse the response content into a Python dictionary in JSON format and return it, expressed as ifresponse.status_code=200,return response.json(); if the response status code is not 200, it means the request failed, print the error message, expressed as print(f"Error fetching data:{response.status_code}-{response.text}"), and return None as the result, expressed as return None.

[0082] Define the save_data_to_file function to save the acquired data to a local file; use the open function to open the file in write mode ('w'), and create it if it does not exist. filename is the file name parameter, expressed as with open(filename,'w') as file; use json.dump to write the data to the file in JSON format, expressed as json.dump(data,file).

[0083] Define the main function main, which uses an infinite loop to continuously execute the data collection and saving process; in each loop, first get data from the API, represented by data=fetch_data_from_api(); if successful (data is not None), get the string representation of the current time in the format of year-month-day_hour-minute-second, represented by timestamp=datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S"); generate a file name containing the current timestamp, represented by filename=f"energy_market_data_{timestamp}.json"; call the save_data_to_file function to save the data to the file, represented by save_data_to_file(data,filename); print the successful saving information, represented by print(f"Data saved to {filename}"); make the program pause for a period of time after each execution of data saving, and then execute it again, represented by time.sleep(0.5 FREQUENCY).

[0084] When the program is run directly (rather than imported into another Python file as a module), the main function is executed; this code checks whether the current script is the main program, and only executes the main() function if it is, which is a common practice to ensure that the program can run as expected when the script is executed directly.

[0085] In this embodiment, the logic flow is:

[0086] Set the data source and parameters, define where (API_URL) and how (via HTTP request HEADERS) to obtain data, the type of data (JSON), and the frequency of data collection (FREQUENCY).

[0087] Define a function for getting data. The fetch_data_from_api function initiates a request to the API through the requests library. If the request is successful, the returned JSON data will be parsed and returned; if it fails, an error message will be printed and None will be returned.

[0088] Define a function for saving data. The save_data_to_file function accepts the collected data and a file name and saves the data in the specified file in JSON format.

[0089] The infinite loop in the main function ensures that the program will repeatedly perform the data acquisition and saving operations at the set frequency (once per hour). Each time through the loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next loop cycle.

[0090] Program entry, the final if __name__="__main__":main() ensures that when the script is executed directly, the above process will be triggered and run.

[0091] Data preprocessing is an important step in data analysis and machine learning. Its purpose is to clean data, convert data formats, and standardize data to better perform subsequent feature extraction and selection. Data preprocessing includes:

[0092] Data cleaning is the first step in data preprocessing. Its main goal is to remove duplicate, invalid, or abnormal data records and handle missing values. Specifically, it includes:

[0093] Remove duplicate data. Use data cleaning tools or write scripts to check whether there are duplicate data records in the dataset. If so, delete the duplicate records and only retain unique data.

[0094] Handle invalid and abnormal data. Invalid data may refer to data that does not meet data specifications or exceeds a reasonable range, while abnormal data may be caused by incorrect data entry or data measurement problems. These data need to be cleaned and processed by deletion, replacement or interpolation methods.

[0095] To handle missing values, there may be some missing data in the dataset. For missing values, you can use interpolation methods such as mean (replacing missing values ​​with the average value of a column of data, which is suitable for numerical data and the data follows a normal distribution), median (using the median instead of missing values ​​may be more robust for skewed data or data with outliers), and nearest neighbor filling (estimate missing values ​​based on the corresponding values ​​of similar records, which is especially suitable for scenarios with continuous variables and strong correlation between data). The specific filling method should be selected based on the characteristics of the actual data and analysis requirements.

[0096] Data standardization is the process of converting data of different dimensions or units to a uniform range. It helps eliminate dimensional differences between data and makes the values ​​of various features comparable. Specifically, it includes:

[0097] Data normalization standardizes the Z-score, centers the data according to the mean and scales it to unit variance, so that the processed data has a mean of 0 and a standard deviation of 1, which facilitates the comparison of features of different scales on the same horizontal line; Min-Max normalization (maximum and minimum value scaling): linearly maps the original data according to its maximum and minimum values ​​so that it falls between [0,1] or [-1,1], which is suitable for features with large numerical range differences.

[0098] Data formatting converts data into a unified format to facilitate subsequent processing and analysis; for example, converting date formats into a unified format, converting numeric formats into a unified unit, and converting text formats into a unified encoding. One-hot encoding: For categorical variables, each category is treated as a new binary feature, with only one category retained as 1 and the rest as 0, effectively solving the problem that the model cannot directly process categorical data. Label encoding: Converting each category of a categorical variable into an integer encoding is suitable for ordered categorical variables or when the model can accept integer encoding.

[0099] Feature extraction and selection are key steps in data preprocessing. Its goal is to extract features related to the analysis target and select the most effective features through certain methods to reduce data dimensionality and redundancy. Specifically, it includes:

[0100] Feature extraction: extracting meaningful features from raw data based on the analysis objectives. For example, for energy projects, the ratio of construction costs to expected returns can be calculated to reflect the efficiency of return on investment. It is also possible to construct a comprehensive score for environmental indicators, combining multiple environmental-related sub-indicators into one indicator that better reflects the overall environmental protection level.

[0101] Feature selection uses statistical methods or machine learning algorithms to select the most effective features from the extracted features; correlation analysis can be used to check the correlation coefficients between features, understand the degree of linear correlation between them, and exclude highly multicollinear features to avoid model overfitting; principal component analysis is used for dimensionality reduction to extract the main components of the data, reducing the dimensions while retaining the main variation information of the data, which helps to simplify the model structure and improve training efficiency.

[0102] Step S2: Obtain satellite images of energy projects, input the satellite images into the image feature network model for feature extraction, and output the environmental features of the energy projects.

[0103] Step S2 specifically includes:

[0104] Figure 2-4In the example, Conv2D represents the standard convolution layer, and the convolution kernel size is 3×3 and 1×1; Strides represents the step size, which takes a value of 1 or 2; Activation function layer (Activation( )), The values ​​are ReLU and Sigmoid; Maxpooling2D represents the maximum pooling layer; Reshape represents the tensor reshaping layer; Concat( , )express , Perform tensor splicing; E Represents the feature maps obtained in the image feature network model, The value range is [1,40]. is an integer.

[0105] like Figure 2 As shown, step S2 specifically includes:

[0106] The satellite image (256, 256, 3) is input into the first standard convolutional layer for convolution operation to obtain the feature map E1. The number of convolution kernels in the first standard convolutional layer is 32, the convolution kernel size is 3×3, and the stride is 2; the feature map E1 is 128×128 with 32 channels; the feature map E1 is input into the first normalized activation layer for batch normalization and activation operations to obtain the feature map E2; the feature map E2 is 128×128 with 32 channels.

[0107] The feature map E2 is input into the second standard convolutional layer for convolution operation to obtain the feature map E3. The number of convolution kernels in the second standard convolutional layer is 64, the convolution kernel size is 3×3, and the stride is 2; the feature map E3 is 64×64 with 64 channels; the feature map E3 is input into the second normalized activation layer for batch normalization and activation operations to obtain the feature map E4; the feature map E4 is 64×64 with 64 channels.

[0108] The feature map E4 is input into the expanded receptive field module for receptive field enhancement operation to obtain the feature map E16, which specifically includes:

[0109] The feature map E4 (expanded receptive field input layer) is input into the first maximum pooling layer for maximum pooling operation to obtain the feature map E5; the pooling window size of the first maximum pooling layer is 2×2; the feature map E5 is 32×32 with 64 channels.

[0110] The feature map E5 is input into the first standard convolution activation layer for convolution activation operation to obtain the feature map E6; the number of convolution kernels in the first standard convolution activation layer is 128, the convolution kernel size is 1×1, and the step size is 1; the feature map E6 is 32×32 with 128 channels; the activation function is ReLU.

[0111] The feature map E5 is input into the first dilated convolution activation layer for dilated convolution operation to obtain the feature map E7; the number of convolution kernels in the first dilated convolution activation layer is 128, the convolution kernel size is 3×3, and the dilation rate Dr (dilation_rate) is 1; the feature map E7 is 32×32 with 128 channels; the activation function is ReLU.

[0112] The feature map E5 is input into the second dilated convolution activation layer for dilated convolution operation to obtain the feature map E8; the number of convolution kernels in the second dilated convolution activation layer is 128, the convolution kernel size is 3×3, and the dilation rate Dr is 6; the feature map E8 is 32×32 with 128 channels; the activation function is ReLU.

[0113] The feature map E5 is input into the third dilated convolution activation layer for dilated convolution operation to obtain the feature map E9; the number of convolution kernels in the third dilated convolution activation layer is 128, the convolution kernel size is 3×3, and the dilation rate Dr is 12; the feature map E9 is 32×32 with 128 channels; the activation function is ReLU.

[0114] The feature map E5 is input into the fourth dilated convolution activation layer for dilated convolution operation to obtain the feature map E10; the number of convolution kernels in the fourth dilated convolution activation layer is 128, the convolution kernel size is 3×3, and the dilation rate Dr is 18; the feature map E10 is 32×32 with 128 channels; the activation function is ReLU.

[0115] The feature map E5 is input to the first global average pooling layer for global average pooling operation to obtain the feature map E11; the feature map E11 is (64,); (64,) means that after global average pooling, the 32×32 two-dimensional array of each channel is simplified to a single value; for 64 channels, a one-dimensional vector containing 64 elements is finally obtained; the dimension of this vector (feature map E11) is (64,), which means that it is one-dimensional and contains 64 values, each value represents the global average activation intensity of the original corresponding channel; the feature map E11 is input to The first tensor reshaping layer performs a tensor reshaping operation to obtain feature map E12; feature map E12 is 1×1 with 64 channels; feature map E12 is input into the second standard convolution activation layer for convolution activation operation to obtain feature map E13; the number of convolution kernels in the second standard convolution activation layer is 128, the convolution kernel size is 1×1, and the step size is 1; feature map E13 is 1×1 with 128 channels; the activation function is ReLU; feature map E13 is input into the first upsampling layer for upsampling operation to obtain feature map E14; feature map E14 is 32×32 with 128 channels.

[0116] The feature maps E6, E7, E8, E9, E10, and E14 are input into the first tensor concatenation layer for tensor concatenation to obtain the feature map E15; the feature map E15 is 32×32 with 768 channels.

[0117] The feature map E15 is input into the third standard convolution activation layer for convolution activation operation to obtain the feature map E16; the number of convolution kernels in the third standard convolution activation layer is 128, the convolution kernel size is 1×1, and the step size is 1; the feature map E16 is 32×32 with 128 channels; the activation function is ReLU.

[0118] The feature map E16 is input into the feature fusion module for multi-feature fusion operation to obtain the feature map E31, which specifically includes:

[0119] The feature map E16 (feature fusion input layer) is input to the second upsampling layer for upsampling operation to obtain feature map E17; feature map E17 is 64×64 with 128 channels; feature map E17 is input to the first linear interpolation matching layer for resizing to match the spatial dimension of the original input feature map (E16), using the bilinear interpolation method to obtain feature map E19; feature map E19 is 32×32 with 128 channels; feature map E19 is input to the third standard convolution layer for convolution operation to obtain feature map E21. The number of convolution kernels in the third standard convolution layer is 128, the convolution kernel size is 1×1, and the stride is 1; feature map E21 is 32×32 with 128 channels.

[0120] The feature map E16 is input to the second maximum pooling layer for maximum pooling operation to obtain feature map E18; the pooling window size of the second maximum pooling layer is 2×2; the feature map E18 is 16×16 with 128 channels; the feature map E18 is input to the second linear interpolation matching layer for resizing to match the spatial dimension of the original input feature map (E16), using the bilinear interpolation method to obtain the feature map E20; the feature map E20 is 32×32 with 128 channels; the feature map E20 is input to the fourth standard convolution layer for convolution operation to obtain feature map E22, the number of convolution kernels in the fourth standard convolution layer is 128, the convolution kernel size is 1×1, and the stride is 1; the feature map E22 is 32×32 with 128 channels.

[0121] The feature maps E16, E21, and E22 are input to the first element-wise addition layer for element-wise addition to obtain a feature map E23. The feature map E23 is 32×32 with 128 channels.

[0122] The feature map E23 is input to the third upsampling layer for upsampling operation to obtain the feature map E24; the feature map E24 is 64×64 with 128 channels; the feature map E24 is input to the third linear interpolation matching layer for resizing to match the spatial dimension of the original input feature map (E23), using the bilinear interpolation method to obtain the feature map E26; the feature map E26 is 32×32 with 128 channels; the feature map E26 is input to the fifth standard convolution layer for convolution operation to obtain the feature map E28. The number of convolution kernels in the fifth standard convolution layer is 128, the convolution kernel size is 1×1, and the stride is 1; the feature map E28 is 32×32 with 128 channels.

[0123] The feature map E23 is input to the third maximum pooling layer for maximum pooling operation to obtain feature map E25; the pooling window size of the third maximum pooling layer is 2×2; the feature map E25 is 16×16 with 128 channels; the feature map E25 is input to the fourth linear interpolation matching layer for resizing to match the spatial dimension of the original input feature map (E23), using the bilinear interpolation method to obtain the feature map E27; the feature map E27 is 32×32 with 128 channels; the feature map E27 is input to the sixth standard convolution layer for convolution operation to obtain feature map E29, the number of convolution kernels in the sixth standard convolution layer is 128, the convolution kernel size is 1×1, and the stride is 1; the feature map E29 is 32×32 with 128 channels.

[0124] The feature maps E23, E28, and E29 are input into the second element-wise addition layer for element-wise addition operation to obtain a feature map E30; the feature map E30 is 32×32 with 128 channels.

[0125] The feature map E30 is input into the fourth standard convolution activation layer for convolution activation operation to obtain the feature map E31; the number of convolution kernels in the fourth standard convolution activation layer is 128, the convolution kernel size is 1×1, and the step size is 1; the feature map E31 is 32×32 with 128 channels; the activation function is ReLU.

[0126] In this embodiment, the above content is specifically implemented by defining a feature fusion function that accepts three parameters: feature_maps, num_filters, and num_layers. Feature_maps represents the input feature map, which usually comes from an intermediate layer of the neural network; num_filters represents the number of filters (or channels) of the convolutional layer used, with a default value of 128; num_layers represents the number of times the superposition operation is repeated, with a default value of 2; the variable Fusion_features = feature_maps is initialized, and its value is set to the input feature_maps. This variable will be used to store and update the feature map during the feature fusion module processing; repeatedly execute the specified number of layers (num_layers), expressed as for_in range(num_layers), and loop through the following steps to enhance the feature map:

[0127] Upsample (enlarge) the current feature map to increase its spatial dimension (height and width). The magnification here is 2, expressed as upsampled=UpSampling2D(size=(2,2))(Fusion_features); Downsample (reduce) the current feature map to reduce its spatial dimension, using maximum pooling with a pooling size of 2, expressed as downsampled=MaxPooling2D(pool_size=(2,2))(Fusion_features); Resize the upsampled feature map to match the spatial dimension of the original input feature map, using a bilinear interpolation method, expressed as upsampled_resized=tf.image.resize(upsampled,[feature _maps.shape[1],feature_maps.shape[2]],method=tf.image.ResizeMethod.BILINEAR); Resize the downsampled feature map to match the spatial dimension of the original input feature map, using a bilinear interpolation method, expressed as downsampled_resized=tf.image.resize(downsam pled,[feature_maps.shape[1],feature_maps.shape[2]],method=tf.image.ResizeMethod.BILINEAR); Use 1×1 convolution layer to adjust the number of channels of upsampled and downsampled feature maps to match num_filters, which helps to integrate feature information of different scales, expressed as upsampled_resized=Conv2D(num_filters,(1,1),padding='same')(upsampled_resized), downsampled_resized=Conv2D(num_filters,(1,1),padding='same')(downsampled_resized); Add the adjusted upsampled feature map, downsampled feature map and current feature map to fuse information of different scales, expressed as Fusion_features=Add()([upsampled_ resized,downsampled_resized,Fusion_features]); Finally, after a specified number of cycles, the fused feature map is subjected to final convolution processing to further refine the features. A 3×3 convolution kernel is used for convolution operation and the ReLU activation function is used to integrate the final feature map. This is expressed as Fusion_features=Conv2D(num_filters,(3,3),padding='same',activation='relu')(Fusion_features). Finally, the processed feature map is returned as returnFusion_features.

[0128] The feature map E31 is input to the fourth upsampling layer for upsampling operation to obtain the feature map E32; the feature map E32 is 64×64 with 128 channels; the feature map E4 and the feature map E32 are input to the second tensor splicing layer for tensor splicing operation to obtain the feature map E33; the feature map E33 is 64×64 with 192 channels.

[0129] The feature map E33 is input into the seventh standard convolutional layer for convolution operation to obtain the feature map E34. The number of convolution kernels in the seventh standard convolutional layer is 256, the convolution kernel size is 3×3, and the stride is 2; the feature map E34 is 32×32 with 256 channels; the feature map E34 is input into the third normalized activation layer for batch normalization and activation operations to obtain the feature map E35; the feature map E35 is 32×32 with 256 channels.

[0130] The feature map E35 is input to the eighth standard convolution layer for convolution operation to obtain the feature map E36. The number of convolution kernels in the eighth standard convolution layer is 512, the convolution kernel size is 3×3, and the stride is 2; the feature map E36 is 16×16 with 512 channels; the feature map E36 is input to the fourth normalized activation layer for batch normalization and activation operations to obtain the feature map E37; the feature map E37 is 16×16 with 512 channels.

[0131] The feature map E37 is input into the second global average pooling layer for global average pooling operation to obtain the feature map E38; the feature map E38 is (512,); the feature map E38 is input into the first fully connected layer and the second fully connected layer for full connection operation, and the environmental features are output. The environmental features include vegetation coverage and terrain type.

[0132] In this embodiment, vegetation cover refers to the distribution of plants on the ground and is often used to describe the density or health of vegetation in a region, such as no vegetation, low vegetation cover, medium vegetation cover, and high vegetation cover. Terrain type describes the physical form of the surface, such as the undulations of the terrain, the direction of rivers, and the location of mountains, such as plains, mountains, and rivers. Extracting these two types of information from a satellite image is actually acquiring features at different levels: from the perspective of vegetation cover, the focus is on the distribution and status of plants on the surface; from the perspective of terrain, the focus is on geographical form, which may affect the distribution of vegetation but is independent of vegetation cover. For example, an area may be a mountain with high vegetation cover or a plain with sparse vegetation. In some cases, terrain type may limit the possibility of vegetation cover (for example, rivers generally do not have vegetation cover, and mountains may have more vegetation cover than plains), but they describe different environmental attributes. Therefore, the image feature network model in this application has two independent output branches, each corresponding to a task: the vegetation cover classification branch, whose output categories may include no cover, low cover, medium cover, and high cover; and the terrain classification branch, whose output categories may include plains, mountains, rivers, etc. The two branches can share the initial convolutional layers but separate at the final fully connected layer, outputting their own classification results. This multi-task learning framework improves data processing efficiency and reduces the risk of overfitting, because the model can learn common features across different tasks.

[0133] In this embodiment, the image feature network model includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, a second normalized activation layer, an expanded receptive field module, a feature fusion module, a fourth upsampling layer, a second tensor splicing layer, a seventh standard convolution layer, a third normalized activation layer, an eighth standard convolution layer, a fourth normalized activation layer, a second global average pooling layer, a first fully connected layer, and a second fully connected layer.

[0134] Step S3: Define the state space, action space, and reward mechanism.

[0135] Step S3 specifically includes:

[0136] State space (S): consists of energy market prices, the state of the policy environment, the performance of the current portfolio, extracted environmental characteristics (such as vegetation cover and terrain type), and project operation data.

[0137] Energy market prices and market price fluctuation data can be predicted using time series analysis of historical price data; policy environment status and the impact of policy changes, such as subsidy changes and tax incentives, can be represented by binary flags or continuous scoring systems; project operation data, including construction costs and operating costs; environmental characteristics, such as vegetation cover and terrain type extracted from satellite images; these states can be represented by encoding continuous and discrete data into vectors.

[0138] In this embodiment, the energy market price is the current market price of each energy source. For each energy source, a floating-point number is used to represent its current market price; the policy environment is the policy support level, such as the subsidy intensity for renewable energy, which is represented by a value between 0 and 1 based on the policy support intensity for each energy source; the project performance is the project capacity and operating cost, which are represented by two values; the environmental characteristics are characteristics based on geographic information, such as average vegetation coverage and major terrain types. Based on the numerical evaluation of terrain and vegetation coverage, corresponding values ​​are given for no coverage, low coverage, medium coverage, and high coverage.

[0139] Action space (A): defines possible investment decisions, each of which corresponds to an adjustment to the portfolio: increasing investment in existing projects, reducing or withdrawing investment in existing projects, or keeping current investment unchanged (as a baseline action); assuming there are four energy types, namely wind, solar, fossil fuel, and nuclear energy projects, for each energy type there are three possible actions: increasing investment, reducing investment, and keeping it unchanged. Therefore, the size of the action space is 3 4 =81. For each energy type, detailed investment decision actions are added, such as investment increase or decrease options ranging from 10% to 100%.

[0140] Reward Mechanism (R): Rewards are defined based on investment returns, risk control, and social responsibility factors (such as carbon emissions and the proportion of renewable energy); Investment Return: The investment return of each energy project is calculated based on market price, production capacity, and operating costs; Risk Control: The diversity of the investment portfolio is evaluated and a more balanced energy mix is ​​rewarded; Social Responsibility: Rewards are calculated based on carbon emissions and the proportion of renewable energy use.

[0141] In this embodiment, the simulation environment based on OpenAI Gym is used to simulate complex energy investment decision-making scenarios, specifically including:

[0142] Import the numpy library, a numerical computing library that facilitates processing arrays, matrices, and other mathematical operations; import the gym library, which is a library developed by OpenAI for creating and testing reinforcement learning agents; introduce the spaces module from the gym library to define the observation space and action space of the environment, expressed as import numpy as np, import gym, and from gym import spaces.

[0143] The constructor of class ComplexEnergyInvestmentEnv(gym.Env):def __init__(self):super(ComplexEnergyInvestmentEnv,self).__init__() __init__ initializes several key parameters of the environment, including:

[0144] n_energy_types indicates that there are four different types of energy; market_price indicates the market price dimension of each energy, which is simplified to a constant. It should actually be dynamic data, which is the price per unit of energy or an abstract representation of the price; policy_support indicates the policy support intensity dimension for each energy, which is simplified to a constant, an abstract representation of the policy support intensity; project_performance indicates the overall performance index of all energy projects, which is two indicators for each energy (capacity and operating cost); environmental_feature: environmental feature dimension, such as vegetation cover and terrain type; n_state_variables: calculates the total number of environmental state variables, the sum of various parameters; the total number of state variables is calculated, including the market price and policy support of each energy, project performance and environmental characteristics, which is actually the sum of the market price and policy support of each energy, plus the value of project performance and the value of environmental characteristics, expressed as def __init__(self):super(ComplexEnergyInves tmentEnv,self).__init__()self.n_energy_types=4;self.market_price=2;self.policy_support=2;self.project_performance=2 self.n_energy_types;self.environmental_feature=2;self.n_state_variables=self.n_energy_types self.market_price+self.n_energy_types self.policy_support+self.project_performance+self.environmental_feature.

[0145] Define the observation space as a Box space ranging from 0 to 1, indicating that each state variable is a continuous floating-point number with a length equal to n_state_variables; define the action space as a MultiDiscrete space, which means that there are 10 discrete investment level choices for each energy source, a total of 4 energy sources, so the action space size is 104; set the maximum number of steps n_steps to 100, and initialize the current step counter (current_step) to 0; call the reset method to initialize the environment state, represented by self.observation_space = spaces.Box(low=np.array([0] self.n _state_variables),high=np.array([1] self.n_state_variables),dtype=np.float32); self.action_space=spaces.MultiDiscrete(

[10] self.n_energy_types); self.n_steps =100; self.current_step=0; self.state=self.reset().

[0146] Define the various components of the environment, step(self, action): execute one step of action and return the next state, reward, whether it is completed, and additional information; reset(self): reset the environment to the initial state; _next_state(self, action): calculate the next state based on the current state and action; _calculate_ reward(self, state, action): calculate the reward based on the state and action; _check_done(self): check whether the end condition is met.

[0147] step method, specifically including:

[0148] It receives an action, then updates the current step counter, calls the _next_state method to calculate the next environment state after executing the action; calculates the reward based on the new state and action by calling the _calculate_reward method; checks whether the termination condition has been met by calling the _check_ done method; updates the environment state to the new state, and returns the new state, reward, completion flag, and an empty additional information dictionary, expressed as def step(self, action): self.current_step+=1; next_state=self._next_state(action); reward=self._calculate_reward(next_state, action); done=self._check_done(); self.state=next_state, return next_state, reward, done,{}.

[0149] The reset method is used to reset the environment, generate a new random initial state, reset the current step counter to zero, and return the new environment state, expressed as def reset(self): self.state = np.random. uniform(low = 0, high = 1, size = (self.n_state_variables,)); self.current_step = 0; return self.state.

[0150] The _next_state method is used to simulate the transition of the environment state after executing an action. First, it copies the current state, then iterates over each energy source, updates the market price and project performance-related state variables based on the effect of the action, and finally ensures that the new state variable value is limited to between 0 and 1, expressed as def_next_ state(self, action): next_state = np.copy(self.state); for i in range (self.n_energy_ types): action_effect = (action[i]-5) / 50.0; next_state[i self.market_price]+=actio n_effect; next_state[self.n_energy_types self.market_price+self.n_energy_type s self.policy_support+i 2]+=action_effect;next_state=np.clip(next_state,0,1);

[0151] return next_state , which modifies the state based on the current action. For each energy type, the effect of the action ( action_effect ) is calculated and applied to the corresponding part of the state. action_effect is based on the difference between the action value and a baseline value (5), and is scaled by 50.

[0152] In the _calculate_reward method, it calculates the return on investment, diversity bonus, carbon penalty, and renewable energy bonus, and adds them together to get the total reward, which includes:

[0153] investment_return means calculating the sum of all energy market price parts as the return on investment; diversity_bonus means calculating the standard deviation of the action. Negative values ​​indicate that the action set with smaller rewards (greater diversity) encourages portfolio diversification; carbon_emission_penalty means using the last element in the state variable related to carbon emissions as the penalty term; renewable_energy_bonus means also using the last element in the state variable (assuming it is related to renewable energy) as the reward term. The above items are added together to get the final reward value, which is expressed as def_calculate_reward(self, state, action): investment_return = np.sum(state[:self.n_energy_types self.market_price]); diversity_bonus=-np.std(action); carbon_emission_penalty=state[-2]; renewable_energy_bonus=state[-1]; reward=investment_return+diversity_bonus-carbon_emission_penalty+renewable_energy_bonus; return reward.

[0154] Finally, the _check_done method checks whether the current step has reached the maximum number of steps. If so, it returns True to indicate the end, otherwise it returns False to indicate that it has not ended, which is expressed as def _check _done(self):returnself.current_step>=self.n_steps. In this embodiment, in addition to the fixed number of steps, the environment can be ended according to specific investment goals or conditions. For example, it ends when the total return rate of the investment portfolio reaches a predetermined target, or when the investment proportion of a certain energy exceeds a certain threshold.

[0155] Step S4: Use deep Q-learning network to simulate the energy investment decision-making process.

[0156] Step S4 specifically includes:

[0157] The core idea of ​​the Deep Q-Network (DQN) is to use deep neural networks to approximate the optimal Q-value function. The Q-value function Q(S, A) represents the expected reward of taking action A in a given state S. By continuously updating the network parameters, DQN aims to learn a strategy that can select the action that maximizes the expected reward in a given state. The network input of DQN is the state vector in the state space, and the output is the Q-value evaluation of each possible action. Based on the environment defined above, a corresponding neural network structure is designed, specifically including:

[0158] The dimension of the input is equal to the dimension of the state space. For the ComplexEnergyInvestmentEnv environment, this will be the length of the state vector. The hidden layer contains multiple fully connected layers, each with dozens to hundreds of neurons. The ReLU activation function is often used to increase the nonlinear fitting ability of the network. The dimension of the output layer is equal to the dimension of the action space, and each output node corresponds to the Q value of an action.

[0159] During DQN training, two technologies, experience replay and target Q network, are used to improve the stability and efficiency of learning. Experience replay means that the transfer (state, action, reward, next state) generated by the interaction between the intelligent agent and the environment is stored in the experience pool; during training, a batch of samples are randomly drawn from the experience pool to update the network, which helps to break the correlation between samples and improve learning efficiency; the target Q network refers to the introduction of a target network to stabilize the update of the Q value. The parameters of the target network are regularly copied from the online network, but do not participate in the gradient descent update.

[0160] In this example, a DQN (Deep Q-Network) agent in deep reinforcement learning is defined. This agent is used to learn how to take actions in a given environment to maximize the cumulative reward. Specifically, it includes:

[0161] Import the required libraries: numpy is used to handle numerical calculations; tensorflow.keras builds and trains neural network models; Sequential and Dense are APIs in Keras, used to create sequential models and fully connected layers respectively; Adam is an optimizer used to update model parameters; deque (double-ended queue) and random modules are used for reinforcement learning memory pool and random selection actions, expressed as import numpy as np, from tensorflow.keras.models import Sequential, from tensorflow.keras.layers import Dense, from tensorflow.keras.optimizers import Adam, from collections import deque, import random.

[0162] Define a class named DQNAgent. The initialization method (init) accepts two parameters: the state space size (state_size) and the action space size (action_size). In the energy investment environment, the state may include multiple indicators such as market prices, policy environment status, project operation data, and environmental characteristics, so state_size will be the sum of the number of all these indicators. In the energy investment environment, if each type of energy project can choose to increase investment, reduce investment, or remain unchanged, and there are four different types of energy projects, then the total number of actions is the combination of the behavioral options for each energy type, expressed as class DQNAgent:def __init__(self,state_size,action_size).

[0163] Save the incoming state and action space sizes as instance variables, expressed as self.state_size=state_size, self.action_size=action_size; initialize a double-ended queue (deque) of fixed length (maxlen=2000) to store the experience replay buffer (memory), which is used to store the actions performed in the environment and their results, expressed as self.memory=deque(maxlen=2000).

[0164] Set some hyperparameters. gamma is the discount factor that determines the emphasis on future rewards. epsilon starts at 1, indicating a full exploration strategy, and it gradually decreases to epsilon_min during training. epsilon_decay is the exploration rate decay coefficient, expressed as self.gamma = 0.95, self.epsilon = 1.0, self.epsilon_min = 0.01, and self.epsilon_decay = 0.995.

[0165] Set the learning rate and create an online Q network (self.model) and a target Q network (self.target_model) by calling the private method _build_model(). Then synchronize the weights of the target model, expressed as self.learning_rate=0.001, and create two neural network models: an online model for selecting actions and a target model for calculating the Q value, expressed as self.model=self._build_model(), self.target_model=self.build_model(). Call the update_target_model method to copy the online model weights to the target model, expressed as self.update_target_model().

[0166] The _build_model method defines the deep Q network structure and uses the Sequential method to build a neural network. It has three dense layers. The first layer receives the state as input and has a ReLU activation function; the second layer is also a hidden layer and uses ReLU activation; the third layer outputs the Q value of each possible action, has no activation function (linear activation), uses mean square error (MSE) as the loss function, and is compiled using the Adam optimizer, expressed as def _build_model(self):model = Sequential(), model.add(Dense(24,input_dim=self.state_size,activation='relu')),

[0167] model.add(Dense(24,activation='relu')),model.add(Dense(self.action_size,activation='linear')),model.compile(loss='mse',optimizer=Adam(lr=self.learning_rate)),return model.

[0168] The update_target_model method is used to periodically copy the weights of the online network to the target network to stabilize the learning process, expressed as def update_target_model(self):self.target_model. set_weights(self.model.get_weights()), which defines a method for copying the weights of the online model to the target model; use get_weights to get the weights from the online model, and use set_weights to set these weights to the target model.

[0169] The remember method adds a five-tuple (state, action, reward, next state, and end flag) to the experience replay buffer (memory), def remember(self,state,action, reward,next_state,done): Add the experience as a tuple to the memory double-ended queue, expressed as self.memory.append((state,action,reward,next_state,done)).

[0170] The act method determines the action to be taken based on the current state, and balances exploration and utilization based on the epsilon-greedy strategy. If a random number is less than or equal to the exploration rate epsilon, an action (exploration) is randomly selected, expressed as if np.random.rand()<=self.epsilon:return random.rand range(self.action_size); otherwise, the online model is used to predict the action value of the current state, and the action corresponding to the maximum value is selected (utilization), act_values=self.model.predict(state), return np.arg max(act_values[0]).

[0171] The replay method randomly samples a batch of data from the experience replay buffer for learning (batch_size samples). The target Q value is calculated based on the reward of each sample and the future maximum Q value, and the online network weights are updated. At the same time, if the conditions are met, the epsilon value is further reduced, including:

[0172] Define the replay method to randomly sample from the experience replay buffer (i.e. memory) and train the model, defreplay(self, batch_size):; Randomly extract a batch (batch_size) of experience samples from the memory, and then perform a loop operation:

[0173] Traverse each experience sample, expressed as for state, action, reward, next_state, done in minibatch:; Calculate the target Q value. If the experience is completed, the target is just the reward; otherwise, it also includes the expected maximum reward of the next state (calculated using the target model), expressed as target = reward, if not done: target = (reward + self.gamma np.amax(self.target_model.predict(next_state)[0])); Use the online model to predict the action value of the current state and set the target Q value to the corresponding action, expressed as target_f=self.model.predict(state), target_f[0] [action]=target; Use the updated target value to train the online model, self.model.fit(state,target _f,epochs=1,verbose=0); If the exploration rate epsilon is higher than the minimum value, reduce the exploration rate according to the decay rate, expressed as if self.epsilon>self.epsilon_min:,self.epsilon = self.epsilon_decay.

[0174] Define the load method to load the model weights, expressed as def load(self,name):, self.model.load_weights(name); define the save method to save the model weights, expressed as def save(self,name):, self.model.save_weights(name).

[0175] In this embodiment, reinforcement learning is applied to a single agent in an energy investment decision-making scenario. A single agent (i.e., a single investment entity) learns through a deep Q-learning network (DQN) how to make the best investment decision based on its own observed state.

[0176] Environmental design defines the state space (S), including energy market prices, policy environment status, project operation data and environmental characteristics; defines the action space (A), which includes investment decisions for each energy type, which can increase, decrease or keep investment unchanged; and designs a reward mechanism (R) to calculate rewards based on investment returns, risk control and social responsibility factors.

[0177] Based on the OpenAI Gym simulation environment, the ComplexEnergyInvestmentEnv class is implemented to simulate the energy investment decision-making process, including initializing environment parameters, defining the observation space and action space, and implementing methods such as step(), reset(), _next_state(), _calculate_reward(), and _check_done().

[0178] The implementation of the DQN agent uses the Deep Q Network (DQN) agent to approximate the optimal Q-value function through a neural network model; initialize the DQNAgent class, configure the state space and action space sizes, experience replay buffer, exploration rate and its decay strategy, etc.; build the Deep Q Network model structure, including input layer, hidden layer and output layer, and train it using experience replay and target Q network technology.

[0179] Example 2

[0180] like Figure 5 As shown, the present invention discloses an energy project investment management system based on artificial intelligence, the system includes:

[0181] The energy project collection module 10 is used to collect energy project data.

[0182] The satellite image acquisition module 20 is used to acquire satellite images of energy projects, input the satellite images into the image feature network model for feature extraction, and output environmental features of the energy projects.

[0183] The environment parameter setting module 30 is used to define the state space, action space and reward mechanism.

[0184] The energy investment process module 40 is used to perform the energy investment decision-making process using a deep Q-learning network.

[0185] As an optional implementation, the energy project collection module 10 of the present invention specifically includes:

[0186] Collect data through API calls and web crawlers, set data crawling frequency, and implement data collection, including:

[0187] Import the libraries needed in Python. import requests is used to initiate HTTP requests; import json is used to process JSON data; import time provides various time-related functions; import datetime is used to process dates and times.

[0188] Set the API URL and set the data type to JSON.

[0189] Design a data collection strategy, set the Accept field in the HTTP request header (Headers), tell the server that the expected response data type is JSON, and identify the client information initiating the request through User-Agent, expressed as HEADERS; define the frequency of data collection.

[0190] Define the function fetch_data_from_api to get data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS; if the response status code meets the requirements, the request is successful, and the .json() method is used to parse the response content into a Python dictionary in JSON format and return it; if the response status code does not meet the requirements, the request fails, and an error message is printed; and None is returned as the result.

[0191] Define the save_data_to_file function to save the obtained data to a local file; use the open function to open the file in write mode ('w') and create it if it does not exist; use json.dump to write the data to the file in JSON format.

[0192] Define the main loop. The loop in the main function repeatedly performs data acquisition and saving operations at a set frequency. In each loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next cycle.

[0193] As an optional implementation manner, the satellite image acquisition module 20 of the present invention specifically includes:

[0194] The first two-standard convolution submodule inputs the satellite image into the first standard convolution layer for convolution operation to obtain a feature map E1; inputs the feature map E1 into the first normalized activation layer for batch normalization and activation operation to obtain a feature map E2; inputs the feature map E2 into the second standard convolution layer for convolution operation to obtain a feature map E3; inputs the feature map E3 into the second normalized activation layer for batch normalization and activation operation to obtain the feature map E4.

[0195] The expanded receptive field submodule is used to input the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16.

[0196] The feature fusion submodule is used to input the feature map E16 into the feature fusion module to perform a multi-feature fusion operation to obtain a feature map E31.

[0197] The upsampling submodule is used to input the feature map E31 into the fourth upsampling layer for upsampling operation to obtain a feature map E32; input the feature map E4 and the feature map E32 into the second tensor splicing layer for tensor splicing operation to obtain a feature map E33.

[0198] The seventh standard convolution submodule is used to input the feature map E33 into the seventh standard convolution layer for convolution operation to obtain a feature map E34; input the feature map E34 into the third normalized activation layer for batch normalization and activation operations to obtain a feature map E35.

[0199] The eighth standard convolution submodule is used to input the feature map E35 into the eighth standard convolution layer for convolution operation to obtain a feature map E36; input the feature map E36 into the fourth normalization activation layer for batch normalization and activation operations to obtain a feature map E37.

[0200] The environmental feature output submodule is used to input the feature map E37 into the second global average pooling layer for global average pooling operation to obtain a feature map E38; input the feature map E38 into the first fully connected layer and the second fully connected layer for full connection operation, and output the environmental features.

[0201] As an optional implementation manner, the expanded receptive field submodule of the present invention specifically includes:

[0202] The feature map E4 is input into the first maximum pooling layer for maximum pooling operation to obtain a feature map E5.

[0203] The feature map E5 is input into the first standard convolution activation layer for convolution activation operation to obtain feature map E6; the feature map E5 is input into the first dilated convolution activation layer for dilated convolution operation to obtain feature map E7; the feature map E5 is input into the second dilated convolution activation layer for dilated convolution operation to obtain feature map E8; the feature map E5 is input into the third dilated convolution activation layer for dilated convolution operation to obtain feature map E9; the feature map E5 is input into the fourth dilated convolution activation layer for dilated convolution operation to obtain feature map E10; the feature map E5 is input into the first global average pooling layer for global average pooling operation to obtain feature map E11.

[0204] The feature map E11 is input into the first tensor reshaping layer for a tensor reshaping operation to obtain a feature map E12; the feature map E12 is input into the second standard convolution activation layer for a convolution activation operation to obtain a feature map E13; the feature map E13 is input into the first upsampling layer for an upsampling operation to obtain a feature map E14.

[0205] The feature map E6, the feature map E7, the feature map E8, the feature map E9, the feature map E10 and the feature map E14 are input into the first tensor splicing layer for tensor splicing operation to obtain the feature map E15; the feature map E15 is input into the third standard convolution activation layer for convolution activation operation to obtain the feature map E16.

[0206] As an optional implementation manner, the feature fusion submodule of the present invention specifically includes:

[0207] The feature map E16 is input into the second upsampling layer for upsampling operation to obtain feature map E17; the feature map E17 is input into the first linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, and a bilinear interpolation method is used to obtain feature map E19; the feature map E19 is input into the third standard convolution layer for convolution operation to obtain feature map E21.

[0208] The feature map E16 is input into the second maximum pooling layer for maximum pooling operation to obtain a feature map E18; the feature map E18 is input into the second linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, and a bilinear interpolation method is used to obtain a feature map E20; the feature map E20 is input into the fourth standard convolution layer for convolution operation to obtain a feature map E22.

[0209] The feature map E16, the feature map E21 and the feature map E22 are input into the first element addition layer for element addition operation to obtain a feature map E23.

[0210] The feature map E23 is input into the third upsampling layer for upsampling operation to obtain feature map E24; the feature map E24 is input into the third linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, and a bilinear interpolation method is used to obtain feature map E26; the feature map E26 is input into the fifth standard convolution layer for convolution operation to obtain feature map E28.

[0211] Input the feature map E23 to the third maximum pooling layer for maximum pooling operation to obtain a feature map E25; input the feature map E25 to the fourth linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, using a bilinear interpolation method to obtain a feature map E27; input the feature map E27 to the sixth standard convolution layer for convolution operation to obtain a feature map E29;

[0212] The feature map E23, the feature map E28 and the feature map E29 are input into the second element addition layer for element addition operation to obtain a feature map E30; the feature map E30 is input into the fourth standard convolution activation layer for convolution activation operation to obtain a feature map E31.

[0213] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An energy project investment management method based on artificial intelligence, characterized by: The method comprises: Step S1: Collect energy project data, including basic attributes, economic indicators, technical characteristics, environmental indicators, and policy orientations; Step S2: Obtaining satellite images of the energy project, inputting the satellite images into an image feature network model for feature extraction, and outputting environmental features of the energy project, wherein the environmental features of the energy project include vegetation coverage and terrain type; The image feature network model includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, a second normalized activation layer, an expanded receptive field module, a feature fusion module, a fourth upsampling layer, a second tensor splicing layer, a seventh standard convolution layer, a third normalized activation layer, an eighth standard convolution layer, a fourth normalized activation layer, a second global average pooling layer, a first fully connected layer and a second fully connected layer; Step S3: Define a state space, an action space, and a reward mechanism. The state space includes energy market prices, policy environment status, current portfolio performance, environmental characteristics of the energy projects, and project operation data. The action space includes actions such as increasing investment, decreasing investment, or maintaining investment unchanged for each energy type. The reward mechanism is set based on investment returns, risk control, and social responsibility factors. Step S4: Use a deep Q-learning network to make energy investment decisions. The input of the deep Q-learning network is the state vector of the state space, and the output is the Q-value evaluation of each action to select the investment action that maximizes the long-term cumulative reward.

2. The energy project investment management method based on artificial intelligence according to claim 1 is characterized in that: The collection of energy project data specifically includes: Collect data through API calls and web crawlers, set data crawling frequency, and implement data collection, including: Import the libraries needed in Python. import requests is used to initiate HTTP requests; import json is used to process JSON data; import time provides various time-related functions; import datetime is used to process date and time; Set the API URL; set the data type to JSON; Design a data collection strategy. Set the Accept field in the HTTP request header to tell the server that the expected response data type is JSON. Use the User-Agent field to identify the client initiating the request, expressed as HEADERS. Define the frequency of data collection. Define the function fetch_data_from_api to get data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS; if the response status code meets the requirements, the request is successful, and the .json() method is used to parse the response content into a Python dictionary in JSON format and return it; if the response status code does not meet the requirements, the request fails, and an error message is printed; and None is returned as the result; Define the save_data_to_file function to save the acquired data to a local file; use the open function to open the file in write mode ('w'), and create it if it does not exist; use json.dump to write the data to the file in JSON format; Define the main loop. The loop in the main function repeatedly performs data acquisition and saving operations at a set frequency. In each loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next cycle.

3. The energy project investment management method based on artificial intelligence according to claim 1 is characterized in that: The step of obtaining satellite images of energy projects, inputting the satellite images into an image feature network model for feature extraction, and outputting environmental features of energy projects specifically includes: Input the satellite image into the first standard convolution layer for convolution operation to obtain a feature map E1; input the feature map E1 into the first normalized activation layer for batch normalization and activation operation to obtain a feature map E2; input the feature map E2 into the second standard convolution layer for convolution operation to obtain a feature map E3; input the feature map E3 into the second normalized activation layer for batch normalization and activation operation to obtain the feature map E4; Inputting the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16; Input the feature map E16 into the feature fusion module to perform a multi-feature fusion operation to obtain a feature map E31; Input the feature map E31 to the fourth upsampling layer for upsampling operation to obtain a feature map E32; input the feature map E4 and the feature map E32 to the second tensor splicing layer for tensor splicing operation to obtain a feature map E33; Inputting the feature map E33 into the seventh standard convolution layer for convolution operation to obtain a feature map E34; inputting the feature map E34 into the third normalized activation layer for batch normalization and activation operation to obtain a feature map E35; Inputting the feature map E35 into the eighth standard convolution layer for convolution operation to obtain a feature map E36; inputting the feature map E36 into the fourth normalization activation layer for batch normalization and activation operation to obtain a feature map E37; The feature map E37 is input into the second global average pooling layer for global average pooling operation to obtain a feature map E38; the feature map E38 is respectively input into the first fully connected layer and the second fully connected layer for full connection operation to output environmental features.

4. The energy project investment management method based on artificial intelligence according to claim 3 is characterized in that: The step of inputting the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16 specifically includes: Input the feature map E4 into the first maximum pooling layer to perform a maximum pooling operation to obtain a feature map E5; The feature map E5 is input into the first standard convolution activation layer for convolution activation operation to obtain feature map E6; the feature map E5 is input into the first dilated convolution activation layer for dilated convolution operation to obtain feature map E7; the feature map E5 is input into the second dilated convolution activation layer for dilated convolution operation to obtain feature map E8; the feature map E5 is input into the third dilated convolution activation layer for dilated convolution operation to obtain feature map E9; the feature map E5 is input into the fourth dilated convolution activation layer for dilated convolution operation to obtain feature map E10; the feature map E5 is input into the first global average pooling layer for global average pooling operation to obtain feature map E11; Inputting the feature map E11 into the first tensor reshaping layer for performing a tensor reshaping operation to obtain a feature map E12; inputting the feature map E12 into the second standard convolution activation layer for performing a convolution activation operation to obtain a feature map E13; inputting the feature map E13 into the first upsampling layer for performing an upsampling operation to obtain a feature map E14; The feature map E6, the feature map E7, the feature map E8, the feature map E9, the feature map E10 and the feature map E14 are input into the first tensor splicing layer for tensor splicing operation to obtain the feature map E15; the feature map E15 is input into the third standard convolution activation layer for convolution activation operation to obtain the feature map E16.

5. The energy project investment management method based on artificial intelligence according to claim 3 is characterized in that: The feature map E16 is input into the feature fusion module to perform a multi-feature fusion operation to obtain a feature map E31, specifically including: Input the feature map E16 to the second upsampling layer for upsampling to obtain a feature map E17; input the feature map E17 to the first linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, using a bilinear interpolation method to obtain a feature map E19; input the feature map E19 to the third standard convolution layer for convolution to obtain a feature map E21; Input the feature map E16 to the second maximum pooling layer for maximum pooling operation to obtain a feature map E18; input the feature map E18 to the second linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, using a bilinear interpolation method to obtain a feature map E20; input the feature map E20 to the fourth standard convolution layer for convolution operation to obtain a feature map E22; Inputting the feature map E16, the feature map E21, and the feature map E22 into a first element addition layer for element addition operation to obtain a feature map E23; Input the feature map E23 to the third upsampling layer for upsampling to obtain a feature map E24; input the feature map E24 to the third linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, using a bilinear interpolation method to obtain a feature map E26; input the feature map E26 to the fifth standard convolution layer for convolution to obtain a feature map E28; Input the feature map E23 to the third maximum pooling layer for maximum pooling operation to obtain a feature map E25; input the feature map E25 to the fourth linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, using a bilinear interpolation method to obtain a feature map E27; input the feature map E27 to the sixth standard convolution layer for convolution operation to obtain a feature map E29; The feature map E23, the feature map E28 and the feature map E29 are input into the second element addition layer for element addition operation to obtain a feature map E30; the feature map E30 is input into the fourth standard convolution activation layer for convolution activation operation to obtain a feature map E31.

6. The energy project investment management system based on artificial intelligence is characterized by: The system comprises: An energy project collection module is used to collect energy project data, including basic attributes, economic indicators, technical characteristics, environmental indicators and policy orientations; A satellite image acquisition module is used to acquire satellite images of energy projects, input the satellite images into an image feature network model for feature extraction, and output environmental features of the energy projects, including vegetation coverage and terrain type; The image feature network model includes a first standard convolution layer, a first normalized activation layer, a second standard convolution layer, a second normalized activation layer, an expanded receptive field module, a feature fusion module, a fourth upsampling layer, a second tensor splicing layer, a seventh standard convolution layer, a third normalized activation layer, an eighth standard convolution layer, a fourth normalized activation layer, a second global average pooling layer, a first fully connected layer and a second fully connected layer; An environmental parameter setting module is used to define a state space, an action space, and a reward mechanism. The state space includes energy market prices, policy environment status, current portfolio performance, environmental characteristics of the energy project, and project operation data. The action space includes actions such as increasing, decreasing, or maintaining investment for each energy type. The reward mechanism is set based on investment returns, risk control, and social responsibility factors. The energy investment process module is used to use a deep Q-learning network to perform energy investment decision-making. The input of the deep Q-learning network is the state vector of the state space, and the output is the Q-value evaluation of each action to select the investment action that maximizes the long-term cumulative reward.

7. The energy project investment management system based on artificial intelligence according to claim 6 is characterized in that: The energy project collection module specifically includes: Collect data through API calls and web crawlers, set data crawling frequency, and implement data collection, including: Import the libraries needed in Python. import requests is used to initiate HTTP requests; import json is used to process JSON data; import time provides various time-related functions; import datetime is used to process date and time; Set the API URL; set the data type to JSON; Design a data collection strategy. Set the Accept field in the HTTP request header to tell the server that the expected response data type is JSON. Use the User-Agent field to identify the client initiating the request, expressed as HEADERS. Define the frequency of data collection. Define the function fetch_data_from_api to get data from the API; use requests.get to send a GET request to API_URL and attach the defined HEADERS; if the response status code meets the requirements, the request is successful, and the .json() method is used to parse the response content into a Python dictionary in JSON format and return it; if the response status code does not meet the requirements, the request fails, and an error message is printed; and None is returned as the result; Define the save_data_to_file function to save the acquired data to a local file; use the open function to open the file in write mode ('w'), and create it if it does not exist; use json.dump to write the data to the file in JSON format; Define the main loop. The loop in the main function repeatedly performs data acquisition and saving operations at a set frequency. In each loop, it first attempts to acquire data. If the acquisition is successful, it generates a file name containing the current timestamp, saves the data to the file, and prints a message confirming that the data has been saved. After that, the program pauses and waits for the next cycle.

8. The energy project investment management system based on artificial intelligence according to claim 6 is characterized in that: The satellite image acquisition module specifically includes: The first two-standard convolution submodule inputs the satellite image into the first standard convolution layer for convolution operation to obtain a feature map E1; inputs the feature map E1 into the first normalized activation layer for batch normalization and activation operation to obtain a feature map E2; inputs the feature map E2 into the second standard convolution layer for convolution operation to obtain a feature map E3; inputs the feature map E3 into the second normalized activation layer for batch normalization and activation operation to obtain the feature map E4; An expanded receptive field submodule, configured to input the feature map E4 into the expanded receptive field module to perform a receptive field enhancement operation to obtain a feature map E16; A feature fusion submodule, configured to input the feature map E16 into the feature fusion module to perform a multi-feature fusion operation to obtain a feature map E31; An upsampling submodule, configured to input the feature map E31 into the fourth upsampling layer for upsampling operation to obtain a feature map E32; input the feature map E4 and the feature map E32 into the second tensor splicing layer for tensor splicing operation to obtain a feature map E33; a seventh standard convolution submodule, configured to input the feature map E33 into the seventh standard convolution layer for convolution operation to obtain a feature map E34; and input the feature map E34 into the third normalized activation layer for batch normalization and activation operation to obtain a feature map E35; an eighth standard convolution submodule, configured to input the feature map E35 into the eighth standard convolution layer for convolution operation to obtain a feature map E36; and input the feature map E36 into the fourth normalization activation layer for batch normalization and activation operation to obtain a feature map E37; The environmental feature output submodule is used to input the feature map E37 into the second global average pooling layer for global average pooling operation to obtain a feature map E38; input the feature map E38 into the first fully connected layer and the second fully connected layer for full connection operation, and output the environmental features.

9. The energy project investment management system based on artificial intelligence according to claim 8 is characterized in that: The expanded receptive field submodule specifically includes: Input the feature map E4 into the first maximum pooling layer to perform a maximum pooling operation to obtain a feature map E5; The feature map E5 is input into the first standard convolution activation layer for convolution activation operation to obtain feature map E6; the feature map E5 is input into the first dilated convolution activation layer for dilated convolution operation to obtain feature map E7; the feature map E5 is input into the second dilated convolution activation layer for dilated convolution operation to obtain feature map E8; the feature map E5 is input into the third dilated convolution activation layer for dilated convolution operation to obtain feature map E9; the feature map E5 is input into the fourth dilated convolution activation layer for dilated convolution operation to obtain feature map E10; the feature map E5 is input into the first global average pooling layer for global average pooling operation to obtain feature map E11; Inputting the feature map E11 into the first tensor reshaping layer for performing a tensor reshaping operation to obtain a feature map E12; inputting the feature map E12 into the second standard convolution activation layer for performing a convolution activation operation to obtain a feature map E13; inputting the feature map E13 into the first upsampling layer for performing an upsampling operation to obtain a feature map E14; The feature map E6, the feature map E7, the feature map E8, the feature map E9, the feature map E10 and the feature map E14 are input into the first tensor splicing layer for tensor splicing operation to obtain the feature map E15; the feature map E15 is input into the third standard convolution activation layer for convolution activation operation to obtain the feature map E16.

10. The energy project investment management system based on artificial intelligence according to claim 8 is characterized in that: The feature fusion submodule specifically includes: Input the feature map E16 to the second upsampling layer for upsampling to obtain a feature map E17; input the feature map E17 to the first linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, using a bilinear interpolation method to obtain a feature map E19; input the feature map E19 to the third standard convolution layer for convolution to obtain a feature map E21; Input the feature map E16 to the second maximum pooling layer for maximum pooling operation to obtain a feature map E18; input the feature map E18 to the second linear interpolation matching layer for resizing to match the spatial dimension of the feature map E16, using a bilinear interpolation method to obtain a feature map E20; input the feature map E20 to the fourth standard convolution layer for convolution operation to obtain a feature map E22; Inputting the feature map E16, the feature map E21, and the feature map E22 into a first element addition layer for element addition operation to obtain a feature map E23; Input the feature map E23 to the third upsampling layer for upsampling to obtain a feature map E24; input the feature map E24 to the third linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, using a bilinear interpolation method to obtain a feature map E26; input the feature map E26 to the fifth standard convolution layer for convolution to obtain a feature map E28; Input the feature map E23 to the third maximum pooling layer for maximum pooling operation to obtain a feature map E25; input the feature map E25 to the fourth linear interpolation matching layer for resizing to match the spatial dimension of the feature map E23, using a bilinear interpolation method to obtain a feature map E27; input the feature map E27 to the sixth standard convolution layer for convolution operation to obtain a feature map E29; The feature map E23, the feature map E28 and the feature map E29 are input into the second element addition layer for element addition operation to obtain a feature map E30; the feature map E30 is input into the fourth standard convolution activation layer for convolution activation operation to obtain a feature map E31.

Images