A dynamic decision-making and optimization method, system, device, electronic device and medium

Abstract

The embodiments of the present application provide a dynamic decision-making and optimization method, system, device, electronic device and medium, which relate to the field of data processing technology. The technical solution includes: obtaining time series data, including: position data, status data and environmental data of the mapped object at multiple consecutive moments. Then, risk detection is performed on the mapped object based on the time series data to obtain a risk detection result. When the risk detection result indicates a first risk level, a first adjustment strategy for the mapped object or the environment in which the mapped object is located is determined based on the time series data through a first reinforcement learning model. When the risk detection result indicates a second risk level, a second adjustment strategy for the mapped object or the environment in which the mapped object is located is determined based on the time series data through a second reinforcement learning model, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model and the second reinforcement learning model are different. The efficiency of obtaining the adjustment strategy after risk detection is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a dynamic decision-making and optimization method, system, device, electronic device and medium. Background Art

[0002] Early warnings of potential future disasters allow personnel to understand impending problems and take proactive countermeasures. Current disaster warning platforms typically only offer risk analysis capabilities. After predicting a potential future disaster, manual adjustments to response strategies are required, resulting in low efficiency. For example, in the aftershock detection scenario, if a disaster warning platform detects a potential aftershock leading to road disruptions, manual rerouting of relief supplies is required, wasting valuable emergency response time. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a dynamic decision-making and optimization method, system, device, electronic device, and medium to improve the efficiency of obtaining adjustment strategies after risk detection. The specific technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a dynamic decision-making and optimization method, the method comprising:

[0005] Acquiring time series data, the time series data including: position data, state data, and environment data of the plotted object at multiple consecutive moments;

[0006] Performing risk detection on the plotted object according to the time series data to obtain a risk detection result;

[0007] When the risk detection result indicates a first risk level, determining a first adjustment strategy for the mapped object or the environment in which the mapped object is located by a first reinforcement learning model based on the time series data;

[0008] When the risk detection result indicates a second risk level, based on the time series data, a second adjustment strategy for the mapped object or the environment in which the mapped object is located is determined by a second reinforcement learning model, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model are different from the model parameters of the second reinforcement learning model.

[0009] Optionally, the method further includes:

[0010] Acquire relevant data of multiple dimensions of the plotted object, wherein the relevant data of the multiple dimensions include: position data, state data, and environment data, and the relevant data of each dimension has a timestamp;

[0011] Merging the relevant data with the same timestamp among the relevant data of the multiple dimensions to obtain merged data;

[0012] generating a spatiotemporal identifier for each piece of merged data, the spatiotemporal identifier including the position data, a timestamp, and an object identifier of the plotted object included in the merged data;

[0013] The acquiring of time series data includes:

[0014] Acquire multiple target spatiotemporal identifiers including the object identifier and having consecutive timestamps, and acquire merged data corresponding to the multiple target spatiotemporal identifiers to obtain the time series data.

[0015] Optionally, after acquiring the time series data, the method further includes:

[0016] For position data at multiple consecutive moments, interpolation points between the position data are determined based on a preset curve interpolation algorithm, where the number of interpolation points is determined based on a preset control accuracy;

[0017] Based on the position data and interpolation points between the position data included in the time series data, a curved movement trajectory of the plotted object is generated.

[0018] Optionally, performing risk detection on the plotted object based on the time series data to obtain a risk detection result includes:

[0019] Obtaining a short-term risk of the plotted object determined based on the time series data using a long short-term memory network;

[0020] Obtaining a mid-term risk of the plotted object determined based on the time series data using a Monte Carlo simulation method;

[0021] Obtaining a long-term risk of the plotted object determined based on the time series data using a Bayesian algorithm;

[0022] The risk detection result is obtained based on at least one of the short-term risk, the medium-term risk, and the long-term risk.

[0023] Optionally, determining a first adjustment strategy for the plotted object or the environment in which the plotted object is located by a first reinforcement learning model based on the time series data includes:

[0024] Generate an initial adjustment strategy set;

[0025] Determining a Pareto optimal solution set under multiple preset objectives from the initial adjustment strategy set through an optimization algorithm;

[0026] The first adjustment strategy is generated based on the time series data and the Pareto optimal solution set through the first reinforcement learning model.

[0027] Optionally, determining a second adjustment strategy for the plotted object or the environment in which the plotted object is located by a second reinforcement learning model based on the time series data includes:

[0028] Generate an initial adjustment strategy set;

[0029] Determining a Pareto optimal solution set under multiple preset objectives from the initial adjustment strategy set through an optimization algorithm;

[0030] The second adjustment strategy is generated based on the time series data and the Pareto optimal solution set through the second reinforcement learning model.

[0031] In a second aspect, an embodiment of the present application provides a dynamic decision-making and optimization system, the system comprising:

[0032] The data collection node is used to collect the position data, status data and environment data of the plotted object at each moment, and send the position data, status data and environment data to the computing node;

[0033] The computing node is configured to obtain time series data, the time series data including position data, state data, and environment data of the plotted object at multiple consecutive moments; perform risk detection on the plotted object based on the time series data, obtain risk detection results, and send the risk detection results to the acceleration node;

[0034] The acceleration node is used to determine, based on the time series data, a first adjustment strategy for the mapping object or the environment in which the mapping object is located, through a first reinforcement learning model when the risk detection result indicates a first risk level; and to determine, based on the time series data, a second adjustment strategy for the mapping object or the environment in which the mapping object is located, through a second reinforcement learning model when the risk detection result indicates a second risk level, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model are different from the model parameters of the second reinforcement learning model.

[0035] Optionally, the system further includes:

[0036] The computing node is further configured to receive the first adjustment strategy or the second adjustment strategy sent by the acceleration node, and send the position data, status data, and environment data of the plotted object after the adjustment strategy is executed to the interactive node;

[0037] The interactive node is configured to display a plotted object based on the received position data, state data, and environment data, receive gesture operation data of the user on the displayed plotted object, and send the gesture operation data to the computing node;

[0038] The computing node is further configured to obtain an updated adjustment strategy based on the gesture operation data, and to send the position data, status data, and environment data of the plotted object after the updated adjustment strategy is executed to the interactive node.

[0039] In a third aspect, an embodiment of the present application provides a dynamic decision-making and optimization device, the device comprising:

[0040] An acquisition module is used to acquire time series data, wherein the time series data includes: position data, state data, and environment data of the plotted object at multiple consecutive moments;

[0041] a detection module, configured to perform risk detection on the plotted object based on the time series data to obtain a risk detection result;

[0042] a determination module, configured to determine, based on the time series data and using a first reinforcement learning model, a first adjustment strategy for the mapped object or the environment in which the mapped object is located when the risk detection result indicates a first risk level;

[0043] The determination module is also used to determine, based on the time series data, a second adjustment strategy for the mapped object or the environment in which the mapped object is located through a second reinforcement learning model when the risk detection result indicates a second risk level, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model are different from the model parameters of the second reinforcement learning model.

[0044] Optionally, the device further includes:

[0045] The acquisition module is further configured to acquire relevant data of multiple dimensions of the plotted object, wherein the relevant data of the multiple dimensions include: position data, state data, and environment data, and the relevant data of each dimension has a timestamp;

[0046] a merging module, configured to merge the related data with the same timestamp in the related data of the multiple dimensions to obtain merged data;

[0047] a generating module, configured to generate a spatiotemporal identifier for each piece of merged data, wherein the spatiotemporal identifier includes the position data, a timestamp, and an object identifier of the plotted object included in the merged data;

[0048] The acquisition module is specifically used to:

[0049] Acquire multiple target spatiotemporal identifiers including the object identifier and having consecutive timestamps, and acquire merged data corresponding to the multiple target spatiotemporal identifiers to obtain the time series data.

[0050] Optionally, the device further includes:

[0051] The determining module is further configured to determine, after acquiring the time series data, interpolation points between the position data at a plurality of consecutive moments based on a preset curve interpolation algorithm, wherein the number of interpolation points is determined according to a preset control accuracy;

[0052] A generating module is used to generate a curved moving trajectory of the plotted object based on the position data and interpolation points between the position data included in the time series data.

[0053] Optionally, the detection module is specifically configured to:

[0054] Obtaining a short-term risk of the plotted object determined based on the time series data using a long short-term memory network;

[0055] Obtaining a mid-term risk of the plotted object determined based on the time series data using a Monte Carlo simulation method;

[0056] Obtaining a long-term risk of the plotted object determined based on the time series data using a Bayesian algorithm;

[0057] The risk detection result is obtained based on at least one of the short-term risk, the medium-term risk, and the long-term risk.

[0058] Optionally, the determining module is specifically configured to:

[0059] Generate an initial adjustment strategy set;

[0060] Determining a Pareto optimal solution set under multiple preset objectives from the initial adjustment strategy set through an optimization algorithm;

[0061] The first adjustment strategy is generated based on the time series data and the Pareto optimal solution set through the first reinforcement learning model.

[0062] Optionally, the determining module is specifically configured to:

[0063] Generate an initial adjustment strategy set;

[0064] Determining a Pareto optimal solution set under multiple preset objectives from the initial adjustment strategy set through an optimization algorithm;

[0065] The second adjustment strategy is generated based on the time series data and the Pareto optimal solution set through the second reinforcement learning model.

[0066] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0067] Memory for storing computer programs;

[0068] The processor is configured to implement the method described in any one of the first aspects when executing a program stored in the memory.

[0069] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0070] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0071] Beneficial effects of the embodiments of the present application:

[0072] The dynamic decision-making and optimization methods, systems, devices, electronic devices, and media provided in the embodiments of the present application can perform risk detection on mapped objects based on time series data, obtain risk detection results, and, based on the different risk levels indicated by the risk detection results, determine adjustment strategies for the mapped objects or the environments in which the mapped objects are located using different reinforcement learning models based on the time series data. Specifically, the embodiments of the present application can automatically generate adjustment strategies in a timely manner after risk detection without requiring human intervention, thereby improving the efficiency of obtaining adjustment strategies after risk detection.

[0073] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0075] Figure 1 A flow chart of a dynamic decision-making and optimization method provided in an embodiment of the present application;

[0076] Figure 2 A flowchart of a method for generating a spatiotemporal identifier provided in an embodiment of the present application;

[0077] Figure 3This is an exemplary schematic diagram of a plotting object display interface provided in an embodiment of the present application;

[0078] Figure 4 An exemplary schematic diagram of a process for generating a three-dimensional space movement trajectory provided in an embodiment of the present application;

[0079] Figure 5 An exemplary schematic diagram of a process for generating an interpolation trajectory provided in an embodiment of the present application;

[0080] Figure 6 A flowchart of a method for determining risk detection results provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of the structure of a dynamic decision-making and optimization system provided in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of the structure of another dynamic decision-making and optimization system provided in an embodiment of the present application;

[0083] Figure 9 A schematic diagram of the structure of a dynamic decision-making and optimization device provided in an embodiment of the present application;

[0084] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0086] In order to improve the efficiency of obtaining adjustment strategies after risk detection, the embodiment of the present application provides a dynamic decision-making and optimization method, which is applied to electronic devices. For example, the electronic device can be a server, desktop computer or laptop computer and other devices with data processing capabilities. The software module that implements the dynamic decision-making and optimization method provided by the embodiment of the present application in the electronic device can be called a deduction engine. Figure 1 As shown, the dynamic decision-making and optimization method provided in the embodiment of the present application includes the following steps:

[0087] S101. Acquire time series data, wherein the time series data includes: position data, state data, and environment data of the plotted object at multiple consecutive moments.

[0088] Location data can be collected through a Geographic Information System (GIS), for example, including longitude, latitude, and elevation.

[0089] Status data can be collected using sensors through the Internet of Things (IoT) technology and is therefore also referred to as IoT data. For example, status data includes location information, operating status, and energy consumption data. Location information can be collected through methods such as the Global Positioning System (GPS) or Radio Frequency Identification (RFID) tags. Operating status includes health status and movement status. Health status indicates whether there are any operational anomalies. For example, health status includes temperature, current, and power. Movement status includes movement speed, acceleration, and steering angle.

[0090] The mapping objects and the environmental data can be set according to the actual application scenario. For example, in a simulation drill scenario, the mapping object can be a transport vehicle, and the environmental data include: whether the various sections of the transport vehicle's planned route are passable and the congestion situation. For another example, in an emergency management scenario, the mapping object can be a rescue vehicle or a trapped person, and the environmental data include whether the various sections of the rescue vehicle's rescue route are passable and the congestion situation, or the environmental data include whether the various sections of the evacuation route of the trapped person are passable and the congestion situation. For another example, in an industrial manufacturing scenario, the mapping object can be the production equipment or processed products included in the production line, and the environmental data include the temperature and humidity of the factory where the production equipment is located, or the environmental data include the operating data of the processing equipment that the processed products need to pass through. For another example, in a smart city scenario, the mapped objects may be vehicles traveling in a preset area, and the environmental data may include the weather in the preset area, road width, whether there are construction sections or traffic accident sections, etc.; or the mapped objects may be public facilities, and the environmental data may include the population density in the area served by the public facilities and the frequency of use of the public facilities, etc.

[0091] Optionally, the time series data may also include other data, such as historical adjustment strategies generated for plotted objects.

[0092] The mapping object in the embodiment of the present application may be one or more. When there are multiple mapping objects, the dynamic decision-making and optimization method provided in the embodiment of the present application may be executed for each mapping object respectively.

[0093] S102: Perform risk detection on the plotted object based on the time series data to obtain risk detection results.

[0094] The specific risk detection method can be found in the description below.

[0095] S103: When the risk detection result indicates a first risk level, determine a first adjustment strategy for the mapped object or the environment in which the mapped object is located through a first reinforcement learning model based on the time series data.

[0096] The first risk level may represent a medium risk, and the first reinforcement learning model may be a model such as Double Q-Learning or Q-learning.

[0097] The adjustment strategy output by the first reinforcement learning model can be determined according to the actual application scenario. For example, in a simulation drill scenario, the first adjustment strategy can be the updated planned driving route of the transport vehicle. For another example, in an emergency management scenario, the first adjustment strategy can be the updated rescue path of the rescue vehicle, or the updated type and quantity of rescue resources loaded on the rescue vehicle, or the updated evacuation path of the trapped persons. For another example, in an industrial manufacturing scenario, the first adjustment strategy can be the updated configuration parameters of the production equipment or the updated processing equipment that the processed products need to pass through. For another example, in a smart city scenario, the first adjustment strategy can be the updated green light duration of each intersection in a preset area, or the updated number or model of public facilities, etc.

[0098] S104: When the risk detection result indicates a second risk level, determine a second adjustment strategy for the mapped object or the environment in which the mapped object is located through a second reinforcement learning model based on the time series data.

[0099] The first risk level is lower than the second risk level, the second risk level may represent a high risk, and the second reinforcement learning model may be a Double Q-Learning or Q-learning model. The adjustment strategy for the output of the second reinforcement learning model may be determined based on the actual application scenario.

[0100] In this embodiment of the present application, the risk detection result may be a numerical value within a preset value range, and the preset value range may include multiple sub-ranges, each of which corresponds to a risk level. The sub-ranges do not intersect, and the union of the sub-ranges constitutes the preset value range. The risk level corresponding to the sub-range to which the risk detection result belongs may be used as the risk level represented by the risk detection result.

[0101] For example, the value range of the risk detection result is [0, 1], where [0, 0.5] corresponds to low risk, (0.5, 0.7] corresponds to medium risk, and (0.7, 1] corresponds to high risk. Assuming that the risk detection result is 0.6, it is determined that the risk detection result indicates medium risk.

[0102] The model parameters of the first reinforcement learning model are different from those of the second reinforcement learning model. The model parameters include the network parameters of each network layer in the reinforcement learning model and the hyperparameters of the reinforcement learning model. The network parameters include weights and biases, and the hyperparameters include the learning rate, discount factor, and exploration rate.

[0103] The learning rate controls the update step size of the reinforcement learning model's model parameters. The learning rate controls how quickly the reinforcement learning model learns new information. The learning rate of the first reinforcement learning model is lower than that of the second. For example, if the learning rate of the first reinforcement learning model is 0.05 and the learning rate of the second reinforcement learning model is 0.15, the first reinforcement learning model will use a more conservative approach to generate the first adjustment strategy, reducing oscillations caused by excessive updates. The second reinforcement learning model will use a more aggressive approach to generate the second adjustment strategy, allowing it to adapt more quickly to environmental changes.

[0104] The discount factor is used to balance the emphasis placed on long-term and immediate gains when the reinforcement learning model makes decisions. The discount factor for the first reinforcement learning model is higher than that for the second reinforcement learning model. For example, the discount factor ranges from [0, 1], with values ​​closer to 1 indicating a greater emphasis on long-term gains. The discount factor for the first reinforcement learning model might be 0.9, while the discount factor for the second reinforcement learning model might be 0.7. This allows the first reinforcement learning model to generate a first adjustment strategy with higher long-term gains as the goal, while the second reinforcement learning model to generate a second adjustment strategy with higher immediate gains as the goal.

[0105] The exploration rate is used to control the balance between exploration and exploitation, specifically, to determine whether the reinforcement learning model favors generating adjustment strategies based on previously learned knowledge. The exploration rate of the first reinforcement learning model is lower than that of the second. For example, the exploration rate ranges from [0, 1], with values ​​closer to 0 indicating a greater bias toward generating adjustment strategies based on previously learned knowledge. The exploration rate of the first reinforcement learning model can be 0.1, while the exploration rate of the second reinforcement learning model can be 0.3. This effectively controls the first reinforcement learning model's bias toward generating adjustment strategies based on previously learned knowledge, improving the first reinforcement learning model's decision-making stability. It also controls the second reinforcement learning model's bias toward exploring new adjustment strategies rather than relying on previously learned knowledge, improving the second reinforcement learning model's adaptability to unknown situations.

[0106] When the risk detection result indicates the first risk level, it means that the mapped object is in a medium-risk scenario. Therefore, the first reinforcement learning model can be used to aim for higher long-term benefits, tend to adopt the learned knowledge, and generate a more conservative adjustment strategy, thereby performing lower-intensity optimization of the mapped object or the environment in which the mapped object is located to ensure the stability of the mapped object.

[0107] When the risk detection result indicates the second risk level, it indicates that the mapped object is in a high-risk scenario, such as an emergency rescue scenario. Therefore, a second reinforcement learning model can be used. Aiming for higher immediate benefits, it tends not to adopt learned knowledge and generates more aggressive strategies, thereby making more intensive adjustments to the mapped object or its environment to achieve the goal of reducing risk. The long-term benefits and immediate benefits can be set according to the needs of the actual application scenario. For example, in an emergency management scenario, the success rate of rescue operations at each rescue destination can be used as the long-term benefit, while the total rescue time and number of rescue personnel at each rescue destination can be used as the immediate benefit.

[0108] The dynamic decision-making and optimization method provided in the embodiments of the present application can perform risk detection on mapped objects based on time series data, obtain risk detection results, and then, based on the different risk levels indicated by the risk detection results, determine an adjustment strategy for the mapped object or its environment using different reinforcement learning models based on the time series data. Specifically, the embodiments of the present application can automatically generate an adjustment strategy in a timely manner after risk detection without requiring human intervention, thereby improving the efficiency of obtaining adjustment strategies after risk detection.

[0109] The following is a detailed description of the dynamic decision-making and optimization method provided in the embodiments of the present application:

[0110] In the embodiment of the present application, the electronic device can obtain the position data, status data, and environment data of the mapped object in real time through a dynamic data pipeline, achieving millisecond-level data injection.

[0111] See Figure 2 , the electronic device may further perform the following steps:

[0112] S201 , obtaining relevant data of multiple dimensions of the plotted object, wherein the relevant data of the multiple dimensions include: position data, state data, and environment data, and the relevant data of each dimension has a timestamp.

[0113] Electronic devices can obtain relevant data of different dimensions from different data sources through message queues such as Kafka. These heterogeneous data can be parsed and formatted to unify the data formats of data of different dimensions, such as converting them into key-value pairs.

[0114] Related data may also include other data, such as predicted data determined based on continuous related data, or historical adjustment strategies generated for plotted objects. The predicted data may be interpolated points in the curve trajectory predicted based on the continuous position data of the plotted object. The determination of the interpolated points is described below.

[0115] S202: Merge the related data with the same timestamp in the related data of multiple dimensions to obtain merged data.

[0116] The timestamp may be the time at which the relevant data is collected. The collection frequency of the relevant data of each dimension may be the same or different. When the collection frequency of the relevant data of each dimension is different, there may be relevant data with different timestamps from the data of other dimensions. The electronic device may delete such relevant data.

[0117] S203 : Generate a spatiotemporal identifier for each piece of merged data, wherein the spatiotemporal identifier includes the position data, a timestamp, and an object identifier of the plotted object included in the merged data.

[0118] The spatiotemporal identifier of each merged data set is unique and can be expressed as a spacetime IDentifier (STID) in the following format: STID = [longitude]_[latitude]_[elevation]_[timestamp]_[object identifier]. Longitude and latitude can include a direction identifier; elevation can include distance units; the accuracy of the timestamp can be set according to actual needs, for example, to the second; and the object identifier can be the model, device ID, or other identifier of the plotted object.

[0119] For example, STID = 111.2222E_33.4444N_50.0m_20250102030405_aa-bb001, where 111.2222E indicates the longitude is 111.2222 degrees east, 33.4444N indicates the latitude is 33.4444 degrees south, 50.0m indicates the elevation is 50 meters, 20250102030405 indicates the timestamp is 3:04:05 on January 2, 2025, and aa-bb001 is the object ID.

[0120] The relevant information of each dimension can be combined into a spatiotemporal matrix. The STID and the spatiotemporal matrix can be stored in an in-memory database such as the Remote Dictionary Server (Redis). One STID can correspond to a row or column of data in the spatiotemporal matrix, thereby achieving high-speed data caching and supporting high-speed data retrieval.

[0121] There may be zero or duplicate values ​​in the spatiotemporal matrix. For example, both location data and status data can include longitude and latitude, and zero or duplicate values ​​have a low contribution to subsequent decision-making and optimization. Therefore, zero and duplicate values ​​can be deleted, thereby reducing the storage space occupied by the spatiotemporal matrix. In addition, the precision of the data in the spatiotemporal matrix can be reduced, for example, controlling the data to no more than four digits after the decimal point, thereby reducing the storage space required for data storage. The data in the spatiotemporal matrix can also be mapped to a lower-dimensional space through a hash function, and the STID can be used as a data index to speed up the retrieval of spatiotemporal data. It is also possible to store only the difference between the predicted data and the relevant data for the predicted data obtained based on the relevant data, thereby reducing the amount of data in the spatiotemporal matrix.

[0122] On this basis, the above-mentioned method of obtaining time series data in S101 can be implemented as follows: obtaining multiple target spatiotemporal identifiers including object identifiers and continuous timestamps, and obtaining merged data corresponding to the multiple target spatiotemporal identifiers to obtain time series data.

[0123] The electronic device can receive a mapping object selected by the user, or determine a preset mapping object, and obtain an object identifier corresponding to the mapping object, as well as determine a deduction time period, and obtain multiple continuous target spatiotemporal identifiers including the object identifier and whose timestamps are in the deduction time period, that is, obtain multiple STIDs. Then, the merged data is retrieved based on the STID to obtain time series data. Among them, the retrieved merged data can be directly sorted in timestamp order as time series data; or, data of a specified dimension can be extracted from the retrieved merged data, such as extracting the location data, status data, and environmental data of the mapping object, and the extracted data can be sorted in timestamp order as time series data.

[0124] Although traditional GIS systems can display the three-dimensional geographic location data of mapped objects, the refresh of the mapped object location usually relies on manual adjustment, cannot be associated with the time dimension, cannot achieve dynamic real-time refresh, and has low efficiency. Commercial simulation software is only used to display two-dimensional plane data, which is somewhat different from the real location and is difficult to meet the needs of high-precision deduction. However, the embodiment of the present application can obtain relevant data of mapped objects from multiple data sources and merge the relevant data of each mapped object at each moment, thereby adding a time dimension to the three-dimensional spatial dimension and assigning a unique space-time identifier, realizing the dynamic injection of multi-dimensional data, and providing a data basis for subsequent deduction and decision-making.

[0125] In the embodiment of the present application, after acquiring the time series data in S101, the electronic device may further use the time series data to deduce the movement trajectory of the plotted object, including the following steps:

[0126] Step 1: For position data at multiple consecutive moments, determine the interpolation points between the position data based on a preset curve interpolation algorithm.

[0127] In order to deduce a smooth movement trajectory based on the position of the plotted object at discrete time points, the embodiment of the present application adopts the Catmull-Rom spline curve interpolation algorithm as the preset curve interpolation algorithm. Alternatively, the embodiment of the present application may also adopt other curve interpolation algorithms, such as cubic spline interpolation or piecewise linear interpolation, etc., which is not specifically limited in the embodiment of the present application.

[0128] Taking the Catmull-Rom spline curve interpolation algorithm as an example, the interpolation point between each of four consecutive position data can be determined from the position data at multiple consecutive moments using the following formula.

[0129] (1);

[0130] in, represents the interpolation point of the incremental time t, 、 、 and is the continuous position data arranged in the order of timestamps, t ∈ [0, 1], t represents t and The time difference between and The time ratio in the time difference between , that is, when t=0, = P1; when t=1, = P2.

[0131] In the embodiment of the present application, the value of t can be determined according to the number of interpolation points. For example, when there is 1 interpolation point, t=0.5 can be taken; when there are 3 interpolation points, t=0.25, 0.5 and 0.75 can be taken, so as to ensure that the interpolation point is and Evenly distributed between.

[0132] The number of interpolation points is determined by the preset control accuracy. See Table 1. Different application scenarios correspond to different control accuracies, and the control accuracy in different scenarios can be set according to actual needs. Different control accuracies can achieve millisecond, second, and minute-level accuracy in the deduced movement trajectory.

[0133] Table 1

[0134] Time granularity Application Scenario Control accuracy millisecond level Flying object trajectory simulation ±0.1 ms Seconds Personnel evacuation trajectory simulation ±0.5 seconds Minute level Transport trajectory simulation ±5 minutes

[0135] Can be based on and The duration of the interval and the control accuracy are determined and For example, in the flight trajectory simulation scenario, and The time period between them is [0.1, 0.5], in milliseconds, and we can calculate (0.5-0.1) / 0.1-1=3, which means the number of interpolation points is 3. For example, in the personnel evacuation trajectory simulation scenario, and The time period between them is [1, 2], in seconds, and we can calculate (2-1) / 0.5-1=1, which means the number of interpolation points is 1. Alternatively, we can use other methods to determine the number of interpolation points based on the control accuracy.

[0136] Step 2: Generate a curved moving trajectory of the plotted object based on the position data included in the time series data and the interpolation points between the position data.

[0137] After generating the curved moving track of the plotted object, the electronic device can also display the three-dimensional model of the plotted object moving along the curved moving track, thereby showing the three-dimensional moving track of the plotted object to the user. Figure 3 As shown, the drawing object display interface may include multiple drawing object buttons, such as a drawing object 1 button, a drawing object 2 button, and a drawing object 3 button. Clicking each button displays the 3D movement trajectory of the corresponding drawing object. For example, if drawing object 1 is a vehicle, clicking the drawing object 1 button displays the vehicle's 3D movement trajectory.

[0138] See also Figure 4 Assume that the plotted object has position data at time t0, t1, t2, and t3 on the time axis, respectively. This position data is recorded as a control point, that is, there are control points P0, P1, P2, and P3. These control points determine the rough shape of the movement trajectory of the plotted object. The electronic device can use the Catmull-Rom spline curve interpolation algorithm to determine the interpolation points based on P0, P1, P2, and P3, and generate a smoother, more refined curved movement trajectory based on P0, P1, P2, P3 and the interpolation points. Then, the three-dimensional model of the plotted object is displayed moving along the curved movement trajectory, thereby showing the three-dimensional spatial movement trajectory of the plotted object to the user.

[0139] It is understandable that the finer the time granularity, the more interpolation points are generated each time, the smoother the curve movement trajectory generated further, and the closer it is to the actual movement trajectory of the plotted object, but the lower the trajectory generation efficiency is.

[0140] See also Figure 5In a scenario with finer temporal granularity, assume that interpolation points 1 to 4 are generated based on the position data of the plotted object, and interpolation trajectory 1 is generated based on the position data and interpolation points 1 to 4. In a scenario with coarser temporal granularity, assume that interpolation points 5 and 6 are generated based on the position data of the plotted object, and interpolation trajectory 2 is generated based on the position data, interpolation points 5, and interpolation points 6. Because interpolation trajectory 1 is generated based on more interpolation points, it is closer to the actual movement trajectory of the plotted object, that is, the error is smaller, and the generation efficiency of interpolation trajectory 1 is lower. Interpolation trajectory 2 is generated based on fewer interpolation points, so there is a large deviation between interpolation trajectory 2 and the actual movement trajectory of the plotted object, that is, the error is larger, and the generation efficiency of interpolation trajectory 2 is higher.

[0141] The embodiment of the present application supports adjusting the preset control accuracy, thereby controlling the interpolation point data generated each time, thereby achieving a balance between trajectory accuracy and generation efficiency.

[0142] The embodiments of the present application can realize the dynamic injection of multi-dimensional data. Since the position data can be associated with the time dimension, the real-time dynamic update of the position can be realized, and a more accurate movement trajectory can be deduced from this.

[0143] In some embodiments of the present application, the risk detection result may indicate the probability of an abnormality occurring in the environment in which the mapped object is located, and / or the probability of an abnormality occurring when the mapped object executes the current strategy in the current environment, where the current strategy may be a preset initial strategy or the most recently obtained first adjustment strategy or second adjustment strategy. Figure 6 The above-mentioned S102 performs risk detection on the environment of the plotted object based on the time series data to obtain the risk detection result, including the following steps:

[0144] S601. Obtain short-term risk of a plotted object determined based on time series data using a long short-term memory network.

[0145] You can input time series data into the Long Short-Term Memory (LSTM) network to get the output of the LSTM. .

[0146] (2);

[0147] in, is the hidden state at time step t, which is also the output of LSTM. represents the Sigmoid function, Represents the data with time step t in the time series data, and are all weight matrices, is the hidden state at time step (t-1), Represents the bias term, and each time step corresponds to the data at a moment in the time series data. The Sigmoid function is an S-shaped function that can map real numbers to the interval (0, 1).

[0148] LSTM is good at capturing the dynamic changes of plotted objects in the short term, so the output of LSTM , which can reflect the short-term risk of the plotted object. For example, in a natural disaster prediction scenario, the LSTM output can indicate whether a natural disaster will recur within one hour after it occurs.

[0149] Among them, the output result It is a value in the range of [0,1]. The closer the value is to 0, the lower the possibility of short-term risk of the plotted object. The closer the value is to 1, the higher the possibility of short-term risk of the plotted object.

[0150] S602. Obtaining the medium-term risk of the plotted object determined based on time series data using a Monte Carlo simulation method.

[0151] The medium-term risk of the plotted object is:

[0152]

[0153]

[0154] in, is the medium-term risk of the plotted object, is the preset number of simulations, represents the mid-term risk assessment function of the simulation results, represents the risk value obtained by the i-th simulation based on time series data, is the preset threshold.

[0155] For example, in a simulation scenario, under certain inventory resources, resource consumption rate, and the amount of resources that can be transported by a transport vehicle in a single trip, the gap between the amount of transport resources from the departure point to the destination and the total resource demand can be simulated based on time series data, that is, It represents the resource gap value obtained from the simulation, thereby obtaining the medium-term risk P, for example, the probability of risk existing within 1 to 24 hours.

[0156] Among them, medium-term risks It is a value in the range of [0,1], representing a probability value. The closer the value is to 0, the lower the possibility that the plotted object has medium-term risk. The closer the value is to 1, the higher the possibility that the plotted object has medium-term risk.

[0157] S603: Obtain the long-term risk of the plotted object determined based on time series data using a Bayesian algorithm.

[0158] The long-term risk of the plotted object is:

[0159]

[0160]

[0161] in, is the probability of event A occurring given event B, which is also the long-term risk of the plotted object. is the probability of event A occurring, is the probability of event B occurring, is the probability of event B occurring when event A occurs, is the probability that event A and event B occur simultaneously.

[0162] In this embodiment of the present application, event A is the target risk event, and event B is the preset pre-event. For example, in an emergency management scenario, event A is the inability of a rescue vehicle to reach the rescue destination, and event B is the continuation or secondary disaster after a natural disaster.

[0163] 、 and , can be obtained by analyzing the time series data. For example, in each experiment, the number of times event A occurs is simulated based on the time series data, and the ratio of the number of times event A occurs to the total number of experiments is used as In each experiment, the number of times event B occurs is simulated based on the time series data, and the ratio of the number of times event B occurs to the total number of experiments is used as In each experiment, the number of times event A and event B occur simultaneously is simulated based on the time series data, and the ratio of the number of times event A and event B occur simultaneously to the total number of experiments is used as .

[0164] By using the Bayesian algorithm, we can analyze the chain reaction between events and predict the evolution of long-term risks, such as the probability of long-term risks in the plotted objects after more than 24 hours. It is a value in the range of [0,1]. The closer the value is to 0, the lower the possibility of long-term risk of the plotted object. The closer the value is to 1, the higher the possibility of long-term risk of the plotted object.

[0165] S604: Obtain risk detection results based on at least one of short-term risk, medium-term risk, and long-term risk.

[0166] In an embodiment of the present application, short-term risks, medium-term risks and long-term risks can all be calculated in parallel, and can all be continuously updated based on time series data, where the update frequency can be the same or different, and the electronic device can obtain the latest short-term risks, medium-term risks and long-term risks when executing S604.

[0167] Optionally, one of short-term risk, medium-term risk and long-term risk may be used as the risk detection result.

[0168] Alternatively, two or three risks can be selected from short-term, medium-term, and long-term risks, and a weighted average of these risks can be calculated. The weights for each risk can be pre-set based on actual needs. This weighted average is then used as the risk detection result. Alternatively, the range to which the weighted average falls within a pre-set range of values ​​can be determined, and the corresponding risk level for this range is used as the risk detection result.

[0169] Through the above method, the embodiment of the present application can perform multi-angle risk prediction on the mapped object, which is suitable for emergency event prediction and long-term risk warning in complex scenarios.

[0170] In some embodiments of the present application, the above-mentioned S103 determines a first adjustment strategy for the plotted object or the environment in which the plotted object is located by using a first reinforcement learning model based on the time series data, including the following steps:

[0171] Step 1: Generate an initial adjustment strategy set.

[0172] Optionally, a group of adjustment strategies may be randomly generated to form an initial adjustment strategy set.

[0173] Taking the emergency management scenario as an example, the adjustment strategy is the scheduling strategy for various rescue resources, including the type and amount of rescue resources allocated to each rescue destination. Rescue resource types include rescue personnel, medical supplies, and search and rescue dogs.

[0174] Step 2: From the initial set of adjustment strategies, an optimization algorithm is used to determine a set of Pareto optimal solutions under multiple preset objectives. A Pareto optimal solution consists of multiple Pareto optimal solutions. A Pareto optimal solution is one that cannot achieve a better objective without compromising another objective. In other words, when an adjustment strategy is Pareto optimal, any modification to the adjustment strategy must compromise at least one objective.

[0175] Multiple preset goals can be set according to the needs of the actual application scenario. Taking the emergency management scenario as an example, the preset goals include the following two:

[0176] Objective 1: Minimize Total Rescue Time. The total rescue time includes the time it takes to transport rescue resources and the time it takes to use them for rescue. The objective function for Objective 1 is:

[0177]

[0178] in, Indicates the total rescue time, Indicates taking the maximum value, It represents the sum of the transportation time of rescue resources to the i-th rescue destination and the time of using rescue resources to provide rescue at the i-th rescue destination. The value of i ranges from [1, N], and N is the total number of rescue destinations.

[0179] Goal 2: Maximize the number of people rescued. The number of people rescued is the total number of disaster victims successfully rescued within the total rescue time. The objective function of Goal 2 is:

[0180]

[0181] in, The number of recipients, is the number of disaster victims successfully rescued at the i-th rescue destination within the total rescue time. The value of i ranges from [1, N], and N is the total number of rescue destinations.

[0182] The above goals are only examples provided in the embodiments of the present application. The preset goals in the emergency management scenario are not limited to these, and the preset goals in other application scenarios are not limited to these.

[0183] At the same time, before determining the Pareto optimal solution, you can also pre-set the constraints of the multi-objective optimization. Constraints can be set according to the needs of the actual application scenario. Taking the emergency management scenario as an example, the constraints include the following three:

[0184] Constraint 1: Resource availability constraints. That is, the quantity of each rescue resource is limited.

[0185]

[0186] in, is the number of the jth type of rescue resources allocated to the i-th rescue destination, is the total number of the jth type of rescue resources, the value range of j is [1, M], and M is the number of types of rescue resources.

[0187] Constraint 2: Time Window Constraints: That is, the rescue time at each rescue destination is limited.

[0188]

[0189] in, is the sum of the time it takes to transport the rescue resources to the i-th rescue destination and the time it takes to use the rescue resources to provide rescue at the i-th rescue destination. is the preset maximum rescue time corresponding to the i-th rescue destination.

[0190] Constraint 3: Resource Allocation Constraints: This means that there is a minimum quantity constraint on the various rescue resources required at each rescue destination.

[0191]

[0192] in, is the number of the jth type of rescue resources allocated to the i-th rescue destination, The minimum number of j-th rescue resources required for the i-th rescue destination.

[0193] The above constraints are only examples provided in the embodiments of this application. The constraints in the emergency management scenario are not limited to this. For example, in the emergency management scenario, there may also be geographical location constraints and traffic condition constraints, and the constraints in other application scenarios are not limited to this.

[0194] Optionally, the optimization algorithm may be an evolutionary algorithm, a particle swarm optimization (PSO) algorithm, or a simulated annealing (SA) algorithm.

[0195] Evolutionary algorithms are metaheuristic methods that search for Pareto optimal solutions by simulating the evolution of a population. Examples of evolutionary algorithms include the non-dominated sorting genetic algorithm II (NSGA-II) or the multi-objective evolutionary algorithm based on decomposition (MOEA / D). NSGA-II is a common multi-objective genetic algorithm that determines Pareto optimal solutions using non-dominated sorting and crowding distance. MOEA / D decomposes a multi-objective optimization problem into single-objective optimization subproblems and then uses an evolutionary algorithm to solve Pareto optimal solutions in parallel.

[0196] Particle swarm optimization algorithm is an optimization algorithm based on swarm intelligence, which searches for Pareto optimal solutions by simulating the foraging behavior of bird flocks.

[0197] The simulated annealing algorithm is a probability-based optimization algorithm that searches for the Pareto optimal solution by simulating the metal annealing process.

[0198] The following takes the optimization algorithm NSGA-Ⅱ as an example to illustrate the process of solving the Pareto optimal solution set:

[0199] 1. Initialize the population, that is, randomly generate a set of initial adjustment strategies to form the initial adjustment strategy set.

[0200] 2. Determine the value of each objective function when the adjustment strategy is each initial adjustment strategy in the adjustment strategy set, for example, the total rescue time and the number of people assisted.

[0201] 3. Based on the objective function values ​​corresponding to each initial adjustment strategy, perform non-dominated sorting on each initial adjustment strategy to obtain multiple non-dominated solutions. Non-dominated sorting involves ranking each initial adjustment strategy according to its non-domination level, which reflects the number of times each solution is dominated by other solutions. A solution is an initial adjustment strategy. Non-dominated solutions are initial adjustment strategies that are not dominated by other initial adjustment strategies. These solutions constitute the Pareto Front, where each non-dominated solution is Pareto optimal. Solution 1 dominates solution 2 if all objective function values ​​of solution 1 are not inferior to those of solution 2, and if at least one objective function value of solution 1 is superior to that of solution 2.

[0202] 4. For each non-dominated solution, calculate the distance between it and its neighboring solutions. For example, use Euclidean distance or Manhattan distance to determine neighboring solutions, and then calculate the distance between solutions. Then, accumulate the calculated distances to obtain the crowding distance of the non-dominated solution. The crowding distance can reflect the density of solutions around the non-dominated solution.

[0203] 5. Based on the non-dominated sorting and congestion, select multiple initial adjustment strategies as parents.

[0204] 6. Perform crossover and mutation operations on the parent generation to generate offspring.

[0205] 7. Merge the parent and offspring generations into a new population.

[0206] 8. Repeat steps 2 to 7 until a preset termination condition is met. For example, the termination condition may be when the number of iterations reaches a preset maximum number of iterations. At this point, the non-dominated solutions obtained in the last iteration are considered the Pareto optimal solution set.

[0207] In an embodiment of the present application, after obtaining a set of Pareto optimal solutions, the objective function value corresponding to each Pareto optimal solution can be displayed, for example, by displaying a scatter plot of the total rescue time and the number of assisted persons for each Pareto optimal solution, and receiving a Pareto optimal solution selected by the decision maker as the first adjustment strategy. For example, if the decision maker places greater emphasis on rescue time, the Pareto optimal solution with the shortest total rescue time can be selected; if the decision maker places greater emphasis on the number of assisted persons, the Pareto optimal solution with the largest number of assisted persons can be selected. Subsequently, the impact of the quantity of different types of resources on the Pareto optimal solution can be analyzed.

[0208] Step 3: Generate a first adjustment strategy based on the time series data and the Pareto optimal solution set through the first reinforcement learning model.

[0209] In an embodiment of the present application, the optimization problem can be modeled as a Markov Decision Process (MDP). Here, time series data is used as the current state, a set of Pareto optimal solutions corresponding to the current state is added to a policy knowledge base, and the weighted sum of the objective function values ​​is used as the immediate benefit. A reinforcement learning model is used to retrieve states matching the current state from the policy knowledge base, and a set of Pareto optimal solutions corresponding to the matching states is obtained, including a set of Pareto optimal solutions corresponding to the current state and a set of Pareto optimal solutions corresponding to states similar to the current state. The obtained set of Pareto optimal solutions is used as a candidate action space. The reinforcement learning model then determines the immediate benefit and long-term benefit obtained when executing each Pareto optimal solution in the candidate action space under the current state, and balances the immediate benefit and long-term benefit based on a discount factor to obtain a value assessment value for the Pareto optimal solution, such as a Q-value. With the goal of maximizing the value assessment value, a Pareto optimal solution is selected from the candidate action space as the first adjustment strategy. At the same time, the current state, each Pareto optimal solution in the candidate action space, and the value assessment value of the Pareto optimal solution can also be recorded in the strategy knowledge base to guide the subsequent action decision-making process of the reinforcement learning model. The embodiment of the present application can realize a closed-loop collaborative mechanism of optimizing the solution set - dynamic decision-making of the reinforcement learning model - iterative update of the strategy knowledge base by combining multi-objective optimization and reinforcement learning models, thereby continuously improving the decision-making ability of the first adjustment strategy.

[0210] After obtaining the first adjustment strategy, the hyperparameters of the first reinforcement learning model may be updated, for example, at least one of the learning rate, discount factor, and exploration rate. The initial values ​​of the hyperparameters can be set based on the requirements of the actual application scenario. The updated hyperparameters are α_{new} = α × (1 + risk level / 10), where α is the hyperparameter before the update. Alternatively, the hyperparameters may be adjusted using other methods. The model parameters of the first reinforcement learning model are then adjusted based on the updated hyperparameters.

[0211] Furthermore, the policy knowledge base can also record the optimization algorithm parameters corresponding to the current state. For example, these parameters include population size, crossover probability, and mutation probability. This allows the candidate action space to also include the optimization algorithm parameters corresponding to the retrieved matching state. The reinforcement learning model can then select an optimization algorithm parameter from the candidate action space as the updated optimization algorithm parameter, so that subsequent optimization algorithms are based on the updated optimization algorithm parameter. The method for selecting optimization algorithm parameters by the reinforcement learning model can be compared to the method for selecting a Pareto optimal solution and will not be further elaborated here.

[0212] Through the above method, the embodiment of the present application can achieve multi-objective optimization, that is, obtain the optimal solution under multiple objectives, which is more suitable for actual application scenarios.

[0213] The method for obtaining the second adjustment strategy is the same as the method for obtaining the first adjustment strategy. Please refer to the above description and will not be repeated here.

[0214] Based on the same inventive concept, corresponding to the above method embodiment, the embodiment of the present application also provides a dynamic decision-making and optimization system, such as Figure 7 As shown, the system includes: a data acquisition node 701, a computing node 702 and an acceleration node 703;

[0215] The data collection node 701 is used to collect the position data, status data and environment data of the plotted object at each time, and send the position data, status data and environment data to the computing node 702. The data collection process can be referred to the description of S101 above and will not be repeated here.

[0216] Computing node 702 is configured to obtain time series data, including the location, status, and environmental data of the plotted object at multiple consecutive moments. Based on the time series data, it performs risk detection on the plotted object, obtains risk detection results, and transmits the risk detection results to acceleration node 703. The risk detection process can be found in the description of S102 above and will not be further elaborated here.

[0217] Acceleration node 703 is configured to determine, based on the time series data, a first adjustment strategy for the plotted object or its environment using a first reinforcement learning model when the risk detection result indicates a first risk level; and to determine, based on the time series data, a second adjustment strategy for the plotted object or its environment using a second reinforcement learning model when the risk detection result indicates a second risk level, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model differ from the model parameters of the second reinforcement learning model. The process of determining the adjustment strategy can be referred to the description of S103 above and will not be repeated here.

[0218] In an embodiment of the present application, the data acquisition node may be an edge gateway, the computing node may be a server or desktop computer or other device, and the acceleration node may be a server or desktop computer or other device with a field-programmable gate array (FPGA) accelerator.

[0219] The dynamic decision-making and optimization system provided in the embodiments of the present application can perform risk detection on mapped objects based on time series data, obtain risk detection results, and, based on the different risk levels indicated by the risk detection results, determine adjustment strategies for the mapped objects or their environments using different reinforcement learning models based on the time series data. Specifically, the embodiments of the present application can automatically generate adjustment strategies in a timely manner after risk detection without requiring human intervention, thereby improving the efficiency of obtaining adjustment strategies after risk detection.

[0220] In some embodiments of the present application, the system may further include an interactive node, for example, a mobile phone or a virtual reality (VR) interactive terminal, or other device with user interaction capabilities.

[0221] The computing node is further configured to receive the first adjustment strategy or the second adjustment strategy sent by the acceleration node, and to send the position data, status data, and environmental data of the plotted object after the adjustment strategy is executed to the interactive node. The computing node may execute the received adjustment strategy, and the position data, status data, and environmental data of the plotted object may change after the adjustment strategy is executed. The computing node may then obtain the changed position data, status data, and environmental data of the plotted object.

[0222] The interactive node is used to display the mapping object based on the received position data, state data and environment data, receive the user's gesture operation data on the displayed mapping object, and send the gesture operation data to the computing node.

[0223] The computing node is further used to obtain an updated adjustment strategy based on the gesture operation data, and to send the position data, status data and environment data of the plotted object after the updated adjustment strategy is executed to the interactive node.

[0224] In an embodiment of the present application, an interactive node can display a three-dimensional model of a plotted object and receive gesture operation data from the user. The gesture operation data includes the gesture operation type and the target object. For example, gesture operation types include pinch-to-zoom, swipe, and select. The target object includes the plotted object and deduction parameters. The deduction parameters include a risk threshold, the weights corresponding to each objective function when calculating rewards, and the hyperparameters of the reinforcement learning model. The risk threshold represents the minimum risk value corresponding to each risk level.

[0225] The time synchronization protocol used between the interaction nodes and the computing nodes can be the Precision Time Protocol-Institute of Electrical and Electronics Engineers 1588v2 (PTP-IEEE 1588v2), which ensures that the data synchronization error between the interaction nodes and the computing nodes is less than 1 millisecond, thereby better adapting to complex decision-making needs.

[0226] The embodiment of the present application supports multimodal interaction, has a VR gesture interaction interface, and supports natural action control such as pinch-to-zoom and sliding operations. Compared with the traditional control method through the mouse and keyboard, it significantly improves the efficiency of human-computer collaboration. Moreover, the traditional deduction system is usually driven by a pre-compiled script. After the user modifies the deduction parameters, it needs to be recompiled. The response delay generally exceeds 5 seconds, and there is a lack of real-time data injection capabilities. It is impossible to perform deduction based on sensor data, etc., resulting in inaccurate deduction results. The embodiment of the present application supports users to adjust the deduction parameters and display the adjusted results in real time. At the same time, it supports the dynamic injection of data from multiple data sources, which improves the accuracy of the deduction results.

[0227] The interactive node can also receive risk detection results sent by the computing node and display the current risk detection results. To more intuitively display the dynamic risk detection results, different risk levels can be set to correspond to different colors and different display effects.

[0228] For example, when the risk level is low, a slowly rotating translucent green sphere is displayed. When the risk level is medium, a pulsating diffuse light ring with a 1 Hz pulse is displayed, and the light ring is yellow. When the risk level is high, flame particles are displayed, and the particle color is red, and a buzzer alarm can also be sounded.

[0229] The following combination Figure 8 , the dynamic decision-making and optimization system involved in the embodiments of the present application is described.

[0230] The software layers involved in the embodiments of the present application include: a data input layer, a spatiotemporal fusion processing engine, and a dynamic deduction decision layer; the hardware layers involved include: a multimodal interaction output layer and a hardware infrastructure layer.

[0231] The data input layer includes real-time data streams from multiple sources, historical adjustment policies generated for plotted objects, and a 3D geographic model of the area where the plotted objects are located. The data streams are accessed via a Kafka message queue, with data being available up to 120 times per second. The historical adjustment policies are stored in a columnar format (Parquet) compressed with a 15:1 compression ratio. The 3D geographic model is stored in 3D Tiles format, with oblique photography at 0.2 meters.

[0232] The spatiotemporal fusion processing engine includes a STID encoder, an interpolation computing cluster, a risk prediction model pool, and a spatiotemporal matrix storage. The STID encoder converts location data from the World Geodetic System 1984 (WGS84) coordinate system to the Earth-Centered, Earth-Fixed (ECEF) coordinate system. The location data is timestamped using BeiDou satellite timing with an accuracy of no more than ±10 nanoseconds. The interpolation computing cluster uses Compute Unified Device Architecture (CUDA) kernels to interpolate location data and generate curved movement trajectories. The risk prediction model pool includes methods such as LSTM and Monte Carlo simulation. The spatiotemporal matrix storage stores the spatiotemporal matrix in Redis, compressed with a 15:1 compression ratio.

[0233] The dynamic deduction decision-making layer consists of a rapid response loop, a deep optimization loop, and a policy knowledge base. The rapid response loop utilizes FPGA acceleration nodes and supports triple modular redundancy (TMR) voting technology to determine the first adjustment strategy. The deep optimization loop utilizes a graphics processing unit (GPU) cluster and Double Q-Learning to determine the second adjustment strategy. The policy knowledge base contains multiple adjustment strategies and can be managed using a distributed version control system (Git).

[0234] The multimodal interaction output layer includes: VR operation terminals, touch screens, and mobile synchronization terminals. VR operation terminals, such as VR helmets, are equipped with somatosensory controllers and a field of view (FOV) of up to 220°. They are used to display mapping objects and receive user gesture operation data. The touch screen can be connected to the computing node, supports 4096 levels of pressure sensitivity, and supports a full-duplex communication protocol based on TCP (WebSocket). It can achieve multi-screen synchronization of data and is used to display mapping objects and receive user gesture operation data. The mobile synchronization terminal has 5G communication capabilities, has low latency, supports the Quick UDP Internet Connections (QUIC) protocol, is connected to the computing node, and is used to display mapping objects. UDP stands for User Datagram Protocol.

[0235] The hardware infrastructure layer includes compute nodes, acceleration nodes, and edge gateways. Compute nodes can be equipped with eight GPUs and bandwidth up to 600 gigabytes per second (GB / s). Acceleration nodes can feature high-performance FPGA accelerator cards designed specifically for data centers, with up to 1.3 million logic cells. Edge gateways can include routers capable of 5G communications, with end-to-end latency of less than 2 milliseconds.

[0236] The embodiment of the present application provides a space-time-data-decision closed-loop system, and innovatively proposes an STID coding system based on space-time integration, which can combine three-dimensional spatial coordinates with nanosecond timestamps and object identifiers to generate a space-time unique ID, thereby adding a time axis dimension based on three-dimensional spatial data, and can achieve millisecond-level retrieval and dynamic update of hundreds of millions of mapped objects.

[0237] The present embodiment also employs a dual-loop optimization mechanism, wherein the fast response loop can generate an adjustment strategy in a more conservative manner and update the hyperparameters of the first reinforcement learning model. The deep optimization loop can generate an adjustment strategy in a more aggressive manner and update the hyperparameters of the second reinforcement learning model. Furthermore, the present embodiment supports multi-objective solution, which can balance the speed and accuracy of strategy generation.

[0238] The embodiment of the present application also supports multimodal interaction, with lower data synchronization latency, significantly improving the efficiency of human-computer collaboration.

[0239] After testing the traditional deduction engine and the deduction engine provided by the embodiment of the present application using a standard simulation test set, it was found that for deduction results, such as curved movement trajectories or scheduling strategies, the accuracy of the traditional deduction engine was 72.3%, while the accuracy of the deduction engine provided by the embodiment of the present application was 89.7%, an improvement of about 24%. For user intervention response time, that is, the time from the user performing a gesture operation to the display of the position data, status data and environmental data of the plotted object after the adjustment strategy is executed, the response time of the traditional deduction engine is 2.4 seconds, and the response time of the deduction engine provided by the embodiment of the present application is 0.28 seconds, an improvement of about 88%. For multi-objective optimization efficiency, the traditional deduction engine uses a single thread to obtain the adjustment strategy, and the embodiment of the present application uses GPU parallel determination of the adjustment strategy, with an improvement of more than 150 times. The maximum concurrent event processing, that is, the maximum amount of related data that can be processed in parallel, is 5,000 items per second for the traditional deduction engine, and 120,000 items per second for the deduction engine provided by the embodiment of the present application, an improvement of more than 24 times.

[0240] Based on the same inventive concept, the embodiment of the present application also provides a dynamic decision-making and optimization device, such as Figure 9 As shown, the device includes: an acquisition module 901, a detection module 902 and a determination module 903;

[0241] The acquisition module 901 is used to acquire time series data, which includes: position data, state data and environment data of the plotted object at multiple consecutive moments;

[0242] A detection module 902 is used to perform risk detection on the plotted object based on the time series data to obtain a risk detection result;

[0243] A determination module 903 is configured to determine, based on the time series data and using a first reinforcement learning model, a first adjustment strategy for the mapped object or the environment in which the mapped object is located when the risk detection result indicates a first risk level;

[0244] Determination module 903 is also used to determine a second adjustment strategy for the mapped object or the environment in which the mapped object is located through a second reinforcement learning model based on time series data when the risk detection result indicates a second risk level, wherein the first risk level is lower than the second risk level, and the model parameters of the first reinforcement learning model are different from the model parameters of the second reinforcement learning model.

[0245] Optionally, the device may further include:

[0246] The acquisition module 901 is further used to acquire relevant data of multiple dimensions of the plotted object, the relevant data of the multiple dimensions including: position data, state data and environment data, and the relevant data of each dimension has a timestamp;

[0247] A merging module is used to merge related data with the same timestamp in related data of multiple dimensions to obtain merged data;

[0248] A generating module, configured to generate a spatiotemporal identifier for each piece of merged data, the spatiotemporal identifier including position data, a timestamp, and an object identifier of a plotted object included in the merged data;

[0249] The acquisition module 901 is specifically used to:

[0250] Acquire multiple target spatiotemporal identifiers including object identifiers and having continuous timestamps, and acquire merged data corresponding to the multiple target spatiotemporal identifiers to obtain time series data.

[0251] Optionally, the device may further include:

[0252] The determination module 903 is further configured to determine, after acquiring the time series data, interpolation points between the position data at a plurality of consecutive moments based on a preset curve interpolation algorithm, wherein the number of interpolation points is determined according to a preset control accuracy;

[0253] The generation module is used to generate a curved movement trajectory of the plotted object based on the position data included in the time series data and the interpolation points between the position data.

[0254] Optionally, the detection module 902 is specifically configured to:

[0255] Obtain the short-term risk of the plotted object determined based on time series data using a long short-term memory network;

[0256] Obtain the medium-term risk of the plotted object determined based on time series data using the Monte Carlo simulation method;

[0257] Obtain the long-term risk of the plotted object determined based on time series data using a Bayesian algorithm;

[0258] A risk detection result is obtained based on at least one of short-term risk, medium-term risk and long-term risk.

[0259] Optionally, the determination module 903 is specifically configured to:

[0260] Generate an initial adjustment strategy set;

[0261] From the initial adjustment strategy set, determine the Pareto optimal solution set under multiple preset objectives through optimization algorithm;

[0262] A first adjustment strategy is generated based on the time series data and the Pareto optimal solution set through the first reinforcement learning model.

[0263] Optionally, the determination module 903 is specifically configured to:

[0264] Generate an initial adjustment strategy set;

[0265] From the initial adjustment strategy set, determine the Pareto optimal solution set under multiple preset objectives through optimization algorithm;

[0266] Through the second reinforcement learning model, a second adjustment strategy is generated based on the time series data and the Pareto optimal solution set.

[0267] The present application also provides an electronic device, such as Figure 10 As shown, it includes a processor 1001 , a communication interface 1002 , a memory 1003 and a communication bus 1004 , wherein the processor 1001 , the communication interface 1002 , and the memory 1003 communicate with each other via the communication bus 1004 .

[0268] Memory 1003, used for storing computer programs;

[0269] The processor 1001 is configured to implement the steps of the method performed by the electronic device in the above method embodiment when executing the program stored in the memory 1003 .

[0270] The communication bus mentioned in the electronic devices mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, only a single thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0271] The communication interface is used for communication between the above electronic device and other devices.

[0272] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0273] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0274] In another embodiment provided in the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned dynamic decision-making and optimization methods are implemented.

[0275] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the dynamic decision-making and optimization methods in the above embodiments.

[0276] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0277] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0278] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, since the system and device embodiments are generally similar to the method embodiments, their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0279] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A dynamic decision-making and optimization method, characterized in that: The method comprises: Acquiring time series data, the time series data including: position data, status data, and environmental data of the plotted object at multiple consecutive moments, the status data including: operating status and energy consumption data; the plotted object including a transport vehicle; and the environmental data including: the passability and congestion status of each road section included in the planned route of the transport vehicle; Performing risk detection on the plotted object based on the time series data to obtain a risk detection result, the risk detection result being used to represent a probability of an abnormality occurring in the environment in which the plotted object is located and / or a probability of an abnormality occurring when the plotted object executes a current strategy in the current environment, the risk detection result being a value within a preset value range, wherein the preset value range includes multiple subranges, each subrange corresponding to a risk level; When the risk detection result indicates a first risk level, determining, based on the time series data, a first adjustment strategy for the mapped object or the environment in which the mapped object is located using a first reinforcement learning model, the first adjustment strategy including an updated planned driving route of the transport vehicle; when the risk detection result indicates a second risk level, determining, based on the time series data, a second adjustment strategy for the mapped object or the environment in which the mapped object is located by a second reinforcement learning model, wherein the second adjustment strategy includes an updated planned driving route of the transport vehicle, the first risk level is lower than the second risk level, model parameters of the first reinforcement learning model are different from model parameters of the second reinforcement learning model, a learning rate of the first reinforcement learning model is lower than a learning rate of the second reinforcement learning model, a discount factor of the first reinforcement learning model is higher than the discount factor of the second reinforcement learning model, and an exploration rate of the first reinforcement learning model is lower than an exploration rate of the second reinforcement learning model; After acquiring the time series data, the method further includes: For position data at multiple consecutive moments, interpolation points between the position data are determined based on a preset curve interpolation algorithm, where the number of interpolation points is determined based on a preset control accuracy; generating a curved movement trajectory of the plotted object based on the position data and interpolation points between the position data included in the time series data; The performing risk detection on the plotted object according to the time series data to obtain a risk detection result includes: Obtaining a short-term risk of the plotted object determined based on the time series data using a long short-term memory network; Obtaining a mid-term risk of the plotted object determined based on the time series data using a Monte Carlo simulation method; Obtaining a long-term risk of the plotted object determined based on the time series data using a Bayesian algorithm; The risk detection result is obtained based on at least one of the short-term risk, the medium-term risk, and the long-term risk.

2. The method according to claim 1, characterized in that The method further comprises: Acquire relevant data of multiple dimensions of the plotted object, wherein the relevant data of the multiple dimensions include: position data, state data, and environment data, and the relevant data of each dimension has a timestamp; Merging the relevant data with the same timestamp among the relevant data of the multiple dimensions to obtain merged data; generating a spatiotemporal identifier for each piece of merged data, the spatiotemporal identifier including the position data, a timestamp, and an object identifier of the plotted object included in the merged data; The acquiring of time series data includes: Acquire multiple target spatiotemporal identifiers including the object identifier and having consecutive timestamps, and acquire merged data corresponding to the multiple target spatiotemporal identifiers to obtain the time series data.

3. The method according to claim 1, characterized in that Determining, based on the time series data, a first adjustment strategy for the plotted object or the environment in which the plotted object is located by a first reinforcement learning model includes: generating an initial adjustment strategy set; determining, from the initial adjustment strategy set, a Pareto optimal solution set under multiple preset objectives by an optimization algorithm; and generating, by the first reinforcement learning model, the first adjustment strategy based on the time series data and the Pareto optimal solution set; And / or, based on the time series data, determining a second adjustment strategy for the plotted object or the environment in which the plotted object is located through a second reinforcement learning model, includes: generating an initial adjustment strategy set; from the initial adjustment strategy set, determining a Pareto optimal solution set under multiple preset goals through an optimization algorithm; and generating the second adjustment strategy based on the time series data and the Pareto optimal solution set through the second reinforcement learning model.

4. A dynamic decision-making and optimization system, characterized in that: The system comprises: A data collection node is configured to collect location data, status data, and environmental data of the mapped objects at each moment, and transmit the location data, status data, and environmental data to the computing node. The status data includes operating status and energy consumption data. The mapped objects include transport vehicles, and the environmental data includes the passability and congestion status of each road section included in the planned route of the transport vehicle. The computing node is configured to obtain time series data, the time series data including: position data, state data, and environment data of the plotted object at multiple consecutive moments; perform risk detection on the plotted object based on the time series data, obtain a risk detection result, and send the risk detection result to the acceleration node, the risk detection result being used to indicate a probability of an abnormality occurring in the environment in which the plotted object is located and / or a probability of an abnormality occurring when the plotted object executes a current strategy in the current environment, the risk detection result being a value within a preset value range, and the preset value range including multiple subranges, each subrange corresponding to a risk level; The acceleration node is configured to determine, based on the time series data, a first adjustment strategy for the mapped object or the environment in which the mapped object is located through a first reinforcement learning model when the risk detection result indicates a first risk level, the first adjustment strategy including the updated planned driving route of the transport vehicle; and, based on the time series data, to determine, through a second reinforcement learning model, a second adjustment strategy for the mapped object or the environment in which the mapped object is located, when the risk detection result indicates a second risk level, the second adjustment strategy including the updated planned driving route of the transport vehicle, the first risk level being lower than the second risk level, the model parameters of the first reinforcement learning model being different from the model parameters of the second reinforcement learning model, the learning rate of the first reinforcement learning model being lower than the learning rate of the second reinforcement learning model, the discount factor of the first reinforcement learning model being higher than the discount factor of the second reinforcement learning model, and the exploration rate of the first reinforcement learning model being lower than the exploration rate of the second reinforcement learning model; The computing node is further configured to, after acquiring the time series data, determine interpolation points between the position data at a plurality of consecutive moments based on a preset curve interpolation algorithm, wherein the number of interpolation points is determined according to a preset control accuracy; and generate a curved movement trajectory of the plotted object based on the position data included in the time series data and the interpolation points between the position data; The computing node is specifically used to obtain the short-term risk of the plotted object determined based on the time series data using a long short-term memory network; obtain the medium-term risk of the plotted object determined based on the time series data using a Monte Carlo simulation method; obtain the long-term risk of the plotted object determined based on the time series data using a Bayesian algorithm; and obtain the risk detection result based on at least one of the short-term risk, the medium-term risk, and the long-term risk.

5. The system according to claim 4, characterized in that The system further comprises: The computing node is further configured to receive the first adjustment strategy or the second adjustment strategy sent by the acceleration node, and send the position data, status data, and environment data of the plotted object after the adjustment strategy is executed to the interactive node; The interactive node is configured to display a plotted object based on the received position data, state data, and environment data, receive gesture operation data of the user on the displayed plotted object, and send the gesture operation data to the computing node; The computing node is further configured to obtain an updated adjustment strategy based on the gesture operation data, and to send the position data, status data, and environment data of the plotted object after the updated adjustment strategy is executed to the interactive node.

6. A dynamic decision-making and optimization device, characterized in that: The device comprises: an acquisition module for acquiring time series data, the time series data including: position data, status data, and environmental data of the plotted object at multiple consecutive moments, the status data including: operating status and energy consumption data; the plotted object includes a transport vehicle; and the environmental data including: the passability and congestion status of each road section included in the planned route of the transport vehicle; a detection module, configured to perform risk detection on the plotted object based on the time series data to obtain a risk detection result, wherein the risk detection result is used to indicate a probability of an abnormality occurring in the environment in which the plotted object is located and / or a probability of an abnormality occurring when the plotted object executes a current strategy in the current environment, wherein the risk detection result is a value within a preset value range, wherein the preset value range includes multiple subranges, each subrange corresponding to a risk level; a determination module configured to determine, based on the time series data and using a first reinforcement learning model, a first adjustment strategy for the mapped object or the environment in which the mapped object is located, when the risk detection result indicates a first risk level, wherein the first adjustment strategy includes an updated planned driving route of the transport vehicle; a determination module, further configured to, when the risk detection result indicates a second risk level, determine, based on the time series data, a second adjustment strategy for the mapped object or the environment in which the mapped object is located, using a second reinforcement learning model, wherein the second adjustment strategy includes an updated planned driving route of the transport vehicle, the first risk level is lower than the second risk level, model parameters of the first reinforcement learning model are different from model parameters of the second reinforcement learning model, a learning rate of the first reinforcement learning model is lower than a learning rate of the second reinforcement learning model, a discount factor of the first reinforcement learning model is higher than the discount factor of the second reinforcement learning model, and an exploration rate of the first reinforcement learning model is lower than an exploration rate of the second reinforcement learning model; The device further comprises: The determining module is further configured to determine, after acquiring the time series data, interpolation points between the position data at a plurality of consecutive moments based on a preset curve interpolation algorithm, wherein the number of interpolation points is determined according to a preset control accuracy; a generating module, configured to generate a curved moving trajectory of the plotted object based on the position data and interpolation points between the position data included in the time series data; The detection module is specifically used to: Obtaining a short-term risk of the plotted object determined based on the time series data using a long short-term memory network; Obtaining a mid-term risk of the plotted object determined based on the time series data using a Monte Carlo simulation method; Obtaining a long-term risk of the plotted object determined based on the time series data using a Bayesian algorithm; The risk detection result is obtained based on at least one of the short-term risk, the medium-term risk, and the long-term risk.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 3 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images