Bandwidth allocation method, device and equipment in digital twinning scene and medium

Abstract

The invention discloses a bandwidth allocation method, device and equipment in a digital twinning scene and a medium, and the method comprises the steps: an intelligent gateway in a digital twinning application system receives access data of a monitoring region, and views data information; storing the access data of the current monitoring area into a cache, and predicting crowd access distribution in the monitoring area in the next scheduling period; according to the predicted crowd access distribution, performing iterative optimization within a preset time by using a reinforcement learning-based method; and selecting an optimal strategy in the iteration process as a bandwidth allocation strategy of the next scheduling period, and issuing the strategy to a sensor and / or an intelligent terminal in the digital twin application system. According to the method, the crowd access distribution condition is predicted, the grid where the hotspot data is located can be found in a targeted manner, and the bandwidth is reasonably distributed in a differentiated manner, so that the real-time performance of digital twinning application is ensured. The method can be widely applied to the technical field of communication.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a bandwidth allocation method, apparatus, device, and medium in a digital twin scenario. Background Art

[0002] With the development of the Internet of Things, 5G, big data, and artificial intelligence, digital twins have become a promising technology for next-generation networks. They can create virtual models to represent physical objects in the real world. These virtual models continuously predict, estimate, and analyze the real-time status of physical entities. With these virtual models, people can achieve precise mapping between physical objects and digital models.

[0003] A digital twin system primarily consists of three components: a physical object, a virtual twin, and a bidirectional data connection between the physical object and the virtual twin. In the real world, physical objects can be equipment, industrial robots, exhibits, construction vehicles, sensors, or complex physical systems. In a digital twin system, the physical object typically provides real-time data to the virtual twin through its behaviors, attributes, and rules, thereby building a virtual model. The virtual twin is a mirror image of the physical object in the digital world, preserving its key characteristics and recording its real-time status. This provides critical support for building virtual digital worlds such as the metaverse, or for assisting in monitoring and optimizing industrial production.

[0004] At the same time, since the essence of digital twins is to construct a multidimensional, multi-physics, and multi-granular mapping from a physical system to a virtual twin, the freshness of data perception is crucial to the real-time performance of the virtual twin. Fresh and accurate data provides the key raw material for digital twin representation, diagnosis, prediction, and decision-making. For sensors associated with a single physical object, when communication latency is unavoidable, greater bandwidth allows sensors to more intensively collect real-time data and upload it to the virtual twin, achieving a lower information age and better supporting the real-time performance of digital twin applications. However, the bandwidth of regional gateways is limited, and it is impossible to provide unlimited bandwidth to sensors corresponding to each physical object. Balancing bandwidth distribution within the gateway's coverage area is a critical issue. Summary of the Invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a bandwidth allocation method, device, equipment and medium for hotspot data in a digital twin scenario.

[0006] The first technical solution adopted by the present invention is:

[0007] A bandwidth allocation method in a digital twin scenario includes the following steps:

[0008] The intelligent gateway in the digital twin application system receives access data from the monitoring area and checks the data information; the data information includes the geographical location information of each monitoring block involved in the source of business data, the access information of the crowd, and the number of data points in each block;

[0009] The access data of the current monitoring area is stored in the cache, and the access distribution of people in the monitoring area in the next scheduling cycle is predicted;

[0010] Based on the predicted visit distribution of the crowd, the reinforcement learning-based method is used to iteratively optimize within a preset time.

[0011] The optimal strategy in the iterative process is selected as the bandwidth allocation strategy for the next scheduling period, and the strategy is sent to the sensors and / or smart terminals in the digital twin application system.

[0012] Furthermore, the digital twin application system covers several monitoring areas, each of which includes multiple monitoring blocks, each of which contains at least one smart terminal or sensor; the digital twin application system also includes a digital twin server, which requires data from at least one sensor or smart terminal in at least one monitoring block as a data source.

[0013] Furthermore, the predicting of the visit distribution of people in the monitoring area in the next scheduling period includes:

[0014] The monitoring area is gridded and the crowd distribution is regarded as a grayscale map. The locally deployed spatiotemporal series prediction model is used to predict the crowd access distribution in the monitoring area in the next scheduling cycle.

[0015] Furthermore, the iterative optimization within a preset time using a reinforcement learning-based method based on the predicted distribution of crowd access includes:

[0016] Using the predicted next-cycle visit distribution as the environmental condition, the initial allocation strategy is output through the existing reinforcement learning model.

[0017] Taking the initial allocation strategy as the iterative starting point, the reinforcement learning model is iterated within a preset time, aiming to maximize the Q value of the initial allocation grid and find a better allocation strategy;

[0018] If the deadline is reached, stop the iteration of the reinforcement learning model, find the optimal strategy in the current iteration process, and output the optimal strategy.

[0019] Furthermore, the method for determining a better allocation strategy is: when the Q value of the initial allocation block of the new strategy is less than the Q value of the initial allocation block of the original strategy, the allocation strategy remains unchanged; when the Q value of the initial allocation block of the new strategy is greater than or equal to the Q value of the initial allocation block of the original strategy, the allocation strategy is the updated allocation strategy.

[0020] Furthermore, the initial allocation strategy is used as the iterative starting point, and the reinforcement learning model is iterated within a preset time to maximize the Q value of the initial allocation grid and find a better allocation strategy, including:

[0021] The initial block is any block in the monitoring area, and then the breadth-first traversal is performed in a clockwise direction. The reward function for each step is:

[0022]

[0023] Where R i (B) is the benefit of bandwidth allocation for block i; N is the distance that a single block can influence within the monitoring area; N i 、N j is the predicted normalized population ratio of block i and block j, λ is the impact discount factor of a single block in the monitoring area, d(i,j) is the Manhattan distance between block i and block j, and S is the set of blocks that have been allocated bandwidth;

[0024] For block i, the benefit function of its allocated bandwidth is:

[0025]

[0026] Where, Represents the minimum bandwidth required for the normal operation of the mth sensor / intelligent terminal in block i;

[0027] The state during the reinforcement learning iteration is represented by the following formula:

[0028] state i =[(x,y),BW,PN,PNA]

[0029] Where (x, y) represents the coordinate value of the current block i, BW represents the remaining available bandwidth up to the current block, PN represents the proportion of people in the remaining unallocated blocks, and PNA represents the proportion of people that can be affected by the remaining unallocated blocks.

[0030] Furthermore, the initial grid Q value of the optimal strategy should be the maximum value among all strategies and the initial strategy during the iteration process. For strategy π, its Q value is expressed using the Bellman equation:

[0031] Q π (s,a)=r(s,a)+γQπ (s′,a′)

[0032] Where s is the state i , s′ is the next state, a is the element in the action space ActionSpace, a′ is the action selected by the next state strategy π, and γ is the reinforcement learning discount factor.

[0033] The second technical solution adopted by the present invention is:

[0034] A bandwidth allocation device in a digital twin scenario, comprising:

[0035] The data receiving module is used for the intelligent gateway in the digital twin application system to receive access data of the monitoring area and view the data information;

[0036] The distribution prediction module is used to store the access data of the current monitoring area into the cache and predict the access distribution of people in the monitoring area in the next scheduling cycle;

[0037] Iterative optimization module, which is used to iteratively optimize within a preset time based on the predicted population access distribution using a reinforcement learning-based method;

[0038] The bandwidth allocation module is used to select the optimal strategy in the iterative process as the bandwidth allocation strategy for the next scheduling period, and send the strategy to the sensors and / or smart terminals in the digital twin application system.

[0039] The third technical solution adopted by the present invention is:

[0040] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a bandwidth allocation method in a digital twin scenario as described above.

[0041] The fourth technical solution adopted by the present invention is:

[0042] A computer-readable storage medium stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a bandwidth allocation method in a digital twin scenario as described above.

[0043] The fifth technical solution adopted by the present invention is:

[0044] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned bandwidth allocation method in a digital twin scenario.

[0045] The beneficial effects of the present invention are as follows: the present invention takes into account the scope of data sources within the monitoring area of the digital twin application, such as sensors and smart terminals. By predicting the distribution of crowd access, the grid where the hotspot data is located can be found in a targeted manner, and the bandwidth can be reasonably and differentially allocated to it, thereby ensuring the real-time performance of the digital twin application; at the same time, considering that the real-time performance of other grids in the digital twin scenario will also affect the user experience of this grid, it is necessary to give reasonable bandwidth to non-hotspot grids while ensuring the differentiated bandwidth of the hotspot grid, so as to optimize the user experience to the greatest extent. The method in the present invention provides an iterative optimization design based on reinforcement learning, so that the final allocation strategy provides a more appropriate service experience for users of each grid while taking into account time complexity and limited bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 This is a flowchart of a bandwidth allocation method in a digital twin scenario according to an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of predicting the visit distribution of a crowd based on a spatiotemporal prediction model in an embodiment of the present invention;

[0049] Figure 3 This is a diagram of the composition of the digital twin application system in an embodiment of the present invention;

[0050] Figure 4 This is a flowchart of the steps of a bandwidth allocation method in a digital twin scenario in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0053] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0054] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0055] Example 1

[0056] like Figure 1 and Figure 4 As shown, this embodiment provides a bandwidth allocation method in a digital twin scenario, which predicts and determines hot data blocks at the communication gateway and makes reasonable and differentiated bandwidth allocation for hot data blocks, so that the real-time performance of digital twin applications is effectively guaranteed while optimizing the user experience. The method specifically includes the following steps:

[0057] S1. The intelligent gateway in the digital twin application system receives the access data of the monitoring area and checks the data information.

[0058] See also Figure 3The digital twin application system of this embodiment covers one or more monitoring areas and includes a digital twin server and several intelligent gateways. The monitoring area includes one or more monitoring blocks, and the monitoring blocks include one or more intelligent terminals or sensors. The intelligent gateway connects one or more intelligent terminals or sensors to the digital twin server. The digital twin server requires data from at least one sensor or intelligent terminal in at least one monitoring block as a data source.

[0059] The data information includes the geographical location information of each block involved in the source of the business data, the crowd access information and the number of data points in each block.

[0060] S2. Store the access data of the current monitoring area into the cache and predict the access distribution of people in the monitoring area in the next scheduling cycle.

[0061] Specifically, the monitoring area is gridded and the crowd distribution is regarded as a grayscale image. The locally deployed spatiotemporal sequence prediction model (such as ConvLSTM) is used to predict the crowd access distribution in the monitoring area in the next scheduling cycle.

[0062] S3. Based on the predicted distribution of people’s visits, use the reinforcement learning-based method to iteratively search for the best result within a preset time.

[0063] The intelligent gateway uses a reinforcement learning-based method to iteratively search for the optimal solution within a fixed timeframe based on information such as the available bandwidth for the next cycle, the predicted access distribution of people within the monitoring area, the geographic location of the block, and the number of data points in the block. As an implementation, the reinforcement learning-based iterative search strategy includes the following steps:

[0064] S31: The gateway obtains the access data of the monitored area and uses the spatiotemporal prediction model to Figure 2 The process shown obtains the crowd access distribution prediction for the next scheduling period.

[0065] S32: Using the predicted distribution of people visiting in the next period as the environmental condition, the initial allocation strategy is output through the existing reinforcement learning model.

[0066] S33: Using the initial allocation strategy as the starting point for iteration, let the reinforcement learning model iterate a finite number of steps within a fixed time, aiming to maximize the Q value of the initial allocation grid and find a better allocation strategy. The initial block is any block in the monitoring area, and the subsequent breadth-first traversal is performed in a clockwise direction. The reward function for each step is:

[0067]

[0068] Among them, R i(B) is the benefit of bandwidth allocation for block i, N is the distance that a single block can affect in the monitoring area, N i , N j is the predicted normalized population ratio of block i and block j, λ is the impact discount factor of a single block in the monitoring area, d(i,j) is the Manhattan distance between block i and block j, and S is the set of blocks that have been allocated bandwidth.

[0069] More specifically, for block i, the benefit function brought to itself by allocating bandwidth is:

[0070]

[0071] in, represents the minimum bandwidth required for the normal operation of the mth sensor / intelligent terminal in block i. The reward function here is discretized to ensure the consistency of device data perception within the block. For each block, its action space is also discrete during the iterative process of reinforcement learning, as follows:

[0072]

[0073] It should be noted that the state in the reinforcement learning iteration process is represented by the following formula:

[0074] state i =[(x,y),BW,PN,PNA]

[0075] Where (x, y) represents the coordinate value of the current block i, BW represents the remaining available bandwidth up to the current block, PN represents the proportion of the population in the remaining unallocated blocks, and PNA represents the proportion of the population that can be affected by the remaining unallocated blocks.

[0076] S34: When the deadline is reached, the reinforcement learning model stops iterating and searches for the optimal strategy in the current iteration. The initial grid Q value of the optimal strategy should be the maximum value among all strategies and the initial strategy in the iteration. For strategy π, its Q value can be expressed by the Bellman equation:

[0077] π π (s,a)=r(s,a)+γQ π (s′,a′)

[0078] Among them, s is the above state i , s′ is the next state, a is the element in the above action space ActionSpace, a′ is the action selected by the next state strategy π, γ is the reinforcement learning discount factor, which is 1 in this scheme;

[0079] S35: Output the optimal strategy and send it to the sensors and smart terminals in the monitoring area.

[0080] S4. Select the optimal strategy in the iterative process as the bandwidth allocation strategy for the next scheduling period, and send the strategy to the sensors and / or smart terminals in the digital twin application system.

[0081] The gateway selects the optimal strategy during the iteration process as the bandwidth allocation strategy for the next scheduling period, and sends the strategy to the sensors and smart terminal devices in each block, and this process ends.

[0082] In summary, the present invention discloses a bandwidth allocation method for hotspot data in a digital twin scenario, including: Step 1: The digital twin application gateway receives the access data of the monitored area and checks the data information, wherein the data information includes the geographical location information of each block involved in the source of the business data, the crowd access information, and the number of data points in each block; Step 2: The gateway stores the access data of the currently monitored area into the cache, and at the same time uses the locally deployed spatiotemporal series prediction model to predict the distribution of crowd access in the monitoring area in the next scheduling cycle; Step 3: The gateway uses the reinforcement learning method to iteratively search for the best within a fixed time based on the available bandwidth of the next cycle, the predicted distribution of crowd access in the monitoring area, the geographical location of the block, and the number of block data points; Step 4: The gateway selects the optimal strategy in the iterative process as the bandwidth allocation strategy for the next scheduling cycle, and sends the strategy to the sensors and smart terminal devices of each block, and the process ends. This method is a bandwidth allocation scheme for hotspot data, which takes into account the scope involved in the data source of the digital twin model, such as sensors and smart terminals. By determining the hot spots of data sources and allocating bandwidth preferentially, the information age of hot spot data on the digital twin model can be reduced, so that the digital twin model can be updated in a timely and effective manner; at the same time, the method of the present invention provides an iterative design of the allocation strategy based on reinforcement learning, so that a relatively fresh differentiated information service age can be achieved under a fixed time limit and high bandwidth utilization.

[0083] Example 2

[0084] This embodiment provides a bandwidth allocation device in a digital twin scenario, including:

[0085] The data receiving module is used for the intelligent gateway in the digital twin application system to receive access data of the monitoring area and view the data information;

[0086] The distribution prediction module is used to store the access data of the current monitoring area into the cache and predict the access distribution of people in the monitoring area in the next scheduling cycle;

[0087] Iterative optimization module, which is used to iteratively optimize within a preset time based on the predicted population access distribution using a reinforcement learning-based method;

[0088] The bandwidth allocation module is used to select the optimal strategy in the iterative process as the bandwidth allocation strategy for the next scheduling period, and send the strategy to the sensors and / or smart terminals in the digital twin application system.

[0089] Since the device is a bandwidth allocation device in a digital twin scenario of an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0090] Example 3

[0091] An embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the following Figure 4 A bandwidth allocation method in a digital twin scenario is shown.

[0092] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0093] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0094] Since the electronic device is an electronic device corresponding to the bandwidth allocation method in a digital twin scenario of an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0095] Example 4

[0096] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 4 A bandwidth allocation method in a digital twin scenario is shown.

[0097] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0098] Since the storage medium is the storage medium corresponding to the bandwidth allocation method in a digital twin scenario of an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0099] Example 5

[0100] In some possible implementations, various aspects of the method of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to execute the steps of the bandwidth allocation method in a digital twin scenario according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" for executing various embodiments can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0101] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0102] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0103] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A bandwidth allocation method in a digital twin scenario, characterized in that: The following steps are involved: The intelligent gateway in the digital twin application system receives access data from the monitoring area and checks the data information; The access data of the current monitoring area is stored in the cache, and the access distribution of people in the monitoring area in the next scheduling cycle is predicted; Based on the predicted visit distribution of the crowd, the reinforcement learning-based method is used to iteratively optimize within a preset time. The optimal strategy in the iterative process is selected as the bandwidth allocation strategy for the next scheduling period, and the strategy is sent to the sensors and / or smart terminals in the digital twin application system.

2. The bandwidth allocation method in a digital twin scenario according to claim 1 is characterized in that: The digital twin application system covers several monitoring areas, each of which includes multiple monitoring blocks, each of which contains at least one smart terminal or sensor; the digital twin application system also includes a digital twin server, which requires data from at least one sensor or smart terminal in at least one monitoring block as a data source.

3. The bandwidth allocation method in a digital twin scenario according to claim 1 is characterized in that: The prediction of the visit distribution of people in the monitoring area in the next scheduling period includes: The monitoring area is gridded and the crowd distribution is regarded as a grayscale map. The locally deployed spatiotemporal series prediction model is used to predict the crowd access distribution in the monitoring area in the next scheduling cycle.

4. The bandwidth allocation method in a digital twin scenario according to claim 1 is characterized in that: The method of iteratively searching for the optimal result within a preset time based on the predicted distribution of visitor numbers by a reinforcement learning method includes: Using the predicted next-cycle visit distribution as the environmental condition, the existing reinforcement learning model outputs the initial allocation strategy. Using the initial allocation strategy as the iterative starting point, the reinforcement learning model iterates within a preset time, aiming to maximize the Q value of the initial allocation grid and find a better allocation strategy. If the deadline is reached, stop the iteration of the reinforcement learning model, find the optimal strategy in the current iteration process, and output the optimal strategy.

5. The bandwidth allocation method in a digital twin scenario according to claim 4 is characterized in that: The method for determining a better allocation strategy is as follows: when the Q value of the initial allocation block of the new strategy is less than the Q value of the initial allocation block of the original strategy, the allocation strategy remains unchanged; when the Q value of the initial allocation block of the new strategy is greater than or equal to the Q value of the initial allocation block of the original strategy, the allocation strategy is the updated allocation strategy.

6. The bandwidth allocation method in a digital twin scenario according to claim 4 is characterized in that: The initial allocation strategy is used as the iterative starting point, and the reinforcement learning model is iterated within a preset time to maximize the Q value of the initial allocation grid and find a better allocation strategy, including: The initial block is any block in the monitoring area, and then the breadth-first traversal is performed in a clockwise direction. The reward function for each step is: Where R i (B) is the benefit of bandwidth allocation for block i; N is the distance that a single block can influence within the monitoring area; N i 、N j is the predicted normalized population ratio of block i and block j, λ is the impact discount factor of a single block in the monitoring area, d(i,j) is the Manhattan distance between blocks i and j, and S is the set of blocks to which bandwidth has been allocated; For block i, the benefit function of its allocated bandwidth is: Where, Represents the minimum bandwidth required for the normal operation of the mth sensor / intelligent terminal in block i; The state during the reinforcement learning iteration is represented by the following formula: state i =[(x,y),BW,PN,PNA] Where (x, y) represents the coordinate value of the current block i, BW represents the remaining available bandwidth up to the current block, PN represents the proportion of people in the remaining unallocated blocks, and PNA represents the proportion of people that can be affected by the remaining unallocated blocks.

7. The bandwidth allocation method in a digital twin scenario according to claim 6 is characterized in that: The initial grid Q value of the optimal strategy should be the maximum value among all strategies and the initial strategy during the iteration process. For strategy π, its Q value is expressed by the Bellman equation: Q π (s,a)=r(s,a)+γQ π (s′,a′) Where s is the state i , s′ is the next state, a is the element in the action space ActionSpace, a′ is the action selected by the next state strategy π, and γ is the reinforcement learning discount factor.

8. A bandwidth allocation device in a digital twin scenario, characterized in that: include: The data receiving module is used for the intelligent gateway in the digital twin application system to receive access data of the monitoring area and view the data information; The distribution prediction module is used to store the access data of the current monitoring area into the cache and predict the access distribution of people in the monitoring area in the next scheduling cycle; Iterative optimization module, which is used to iteratively optimize within a preset time based on the predicted population access distribution using a reinforcement learning-based method; The bandwidth allocation module is used to select the optimal strategy in the iterative process as the bandwidth allocation strategy for the next scheduling period, and send the strategy to the sensors and / or smart terminals in the digital twin application system.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

Images