Space computing automatic networking method, device, equipment, medium and product

Abstract

The invention discloses an automatic networking method, device and equipment for spatial computing, a medium and a product, and relates to the technical field of computers, the method comprises the following steps: constructing a super object model, and performing clustering analysis on at least two pieces of equipment based on the super object model to obtain an initial network topology structure; acquiring a current equipment state of equipment corresponding to the initial network topology structure, inputting the current equipment state into a pre-trained reinforcement learning strategy model, and outputting to obtain a first strategy; updating the initial network topology structure according to a first strategy to obtain a first network topology structure; and determining the network performance of the first network topology structure, and if the network performance reaches a preset performance threshold, determining that the first network topology structure is a target network topology structure. According to the invention, the overall performance and resource utilization efficiency of the network are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to methods, devices, equipment, media and products for automatic networking of spatial computing. Background Art

[0002] With the rapid development of IoT technology, the number of IoT devices and application scenarios have shown an explosive growth trend, which has brought unprecedented challenges to network management. The traditional static networking method, as a networking method based on a fixed network architecture, preset device connection relationships and resource allocation strategies, has been unable to meet the needs of the current complex and changeable network environment. The static networking method determines the network topology, device connection relationship and resource allocation scheme at the beginning of network design, and these settings usually remain unchanged during network operation. However, with the continuous increase and movement of IoT devices and the continuous change of business needs, the limitations of the static networking method have gradually emerged. Its fixedness and immutability make the network unable to respond promptly and effectively when facing dynamic changes such as device addition and reduction, location change or business demand adjustment, thus affecting the overall performance and reliability of the network. These problems make the static networking method appear to be powerless in dealing with the current complex and changeable dynamic environment, unable to meet the needs of the rapid growth of IoT devices and the diversification of application scenarios, resulting in poor network performance in the dynamic networking environment of IoT devices. Summary of the invention

[0003] The main purpose of this application is to provide a spatial computing automatic networking method, device, equipment, medium and product, aiming to solve the technical problem of poor network performance in the dynamic networking environment of Internet of Things devices.

[0004] To achieve the above objectives, the present application proposes a spatial computing automatic networking method, comprising:

[0005] Constructing a super object model, and performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices;

[0006] Obtaining the current device state of the device corresponding to the initial network topology structure, inputting the current device state into a pre-trained reinforcement learning strategy model, and outputting a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure;

[0007] The initial network topology structure is updated according to the first strategy to obtain a first network topology structure;

[0008] The network performance of the first network topology is determined, and if the network performance reaches a preset performance threshold, the first network topology is determined as the target network topology.

[0009] In one embodiment, the steps of constructing a super object model include:

[0010] Generate attribute vectors based on the hardware specifications and dynamic status of the device;

[0011] Generate a three-dimensional vector according to the spatial position information of the device;

[0012] A metadata field is defined for each device, and a metadata vector is generated based on the metadata field;

[0013] A super object model is constructed according to the attribute vector, the three-dimensional vector, the metadata vector, and the preset time series, wherein the time series represents the data differences generated by the attribute vector, the three-dimensional vector, and the metadata vector respectively changing over time.

[0014] In one embodiment, the step of performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure includes:

[0015] Based on the super object model, at least two devices are grouped using a preset clustering algorithm to obtain at least one subnet, wherein the clustering algorithm groups devices with similar functions or close spatial locations into the same subnet according to the super object model;

[0016] For each subnet, a first connection relationship between all devices belonging to the subnet is determined, and the subnet is structurally optimized according to the first connection relationship, wherein the initial network topology structure includes each subnet after structural optimization.

[0017] In one embodiment, the step of updating the initial network topology structure according to the first strategy to obtain the first network topology structure includes:

[0018] If the initial network topology structure has multiple subnets, determine whether there is a policy conflict between the first policies corresponding to each subnet;

[0019] If there is no policy conflict between the first policies corresponding to each subnet, then for each subnet, the network topology structure of the subnet is updated according to the first policy corresponding to each subnet to obtain a first network topology structure.

[0020] In one embodiment, after the step of determining whether there is a policy conflict between the first policies corresponding to each subnet, the method further includes:

[0021] If there are at least two subnets corresponding to the first policies that have policy conflicts, the subnets with policy conflicts are determined as conflicting subnets;

[0022] Obtaining a global policy model sent by a preset central control unit, wherein the central control unit determines a global network state according to the subnet states of all subnets, and constructs a global policy model according to the global network state;

[0023] Determine a second decision for the conflicting subnet according to the global policy model, and update the network topology structure of the conflicting subnet according to the second decision to obtain an updated conflicting subnet;

[0024] For the subnets without policy conflicts, the network topology structure is updated according to the first policy corresponding to the subnets without policy conflicts to obtain the second subnets;

[0025] The first network topology is determined according to the updated conflicting subnet and / or the second subnet.

[0026] In one embodiment, the spatial computing automatic networking method further includes:

[0027] The super object model at a preset initial moment is defined as a state vector, an action vector is defined according to the adjustment strategy between devices, and a strategy model is defined based on the mapping relationship between the state vector and the action vector;

[0028] Determine the network performance under the state vector and the action vector, and define a reward function based on the network performance, wherein the reward function is used to evaluate the network performance of the action vector under the state vector;

[0029] The learning algorithm of the strategy model is trained according to the reward function to obtain the reinforcement learning strategy model.

[0030] In addition, to achieve the above purpose, the present application also proposes a spatial computing automatic networking device, which includes:

[0031] A super object model generation module is used to construct a super object model and to perform cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices;

[0032] A strategy generation module obtains the current device state of the device corresponding to the initial network topology structure, inputs the current device state into a pre-trained reinforcement learning strategy model, and outputs a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained based on the super object model and the initial network topology structure;

[0033] A network topology generation module updates the initial network topology structure according to the first strategy to obtain a first network topology structure;

[0034] The performance detection module determines the network performance of the first network topology structure, and if the network performance reaches a preset performance threshold, the first network topology structure is determined to be the target network topology structure. In addition, to achieve the above purpose, the present application also proposes a spatial computing automatic networking device, the device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program is configured to implement the steps of the spatial computing automatic networking method as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the spatial computing automatic networking method as described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a product, which is a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the spatial computing automatic networking method as described above.

[0037] One or more technical solutions proposed in this application have at least the following technical effects:

[0038] The present application constructs a super object model, wherein the super object model includes the hardware specifications, dynamic states, and spatial locations of all preset devices. By abstracting and integrating device information, the present application forms a virtual resource pool to provide basic data support for subsequent network optimization and decision-making; clustering analysis is performed on at least two devices based on the super object model to obtain an initial network topology structure to optimize network performance and resource utilization; the current device state of the device corresponding to the initial network topology structure is obtained, and the current device state is input into a pre-trained reinforcement learning strategy model, and the first strategy is output to enable it to select the optimal action according to the current network state to maximize long-term rewards, such as improving network performance and reducing resource consumption, etc., wherein the reinforcement learning strategy model is a strategy model constructed and trained based on the super object model and the initial network topology structure; the initial network topology structure is updated according to the first strategy to obtain a first network topology structure, and the network performance of the first network topology structure is determined. If the network performance reaches a preset performance threshold, the first network topology structure is determined as the target network topology structure, and the network topology structure that meets the network performance requirements is gradually found through continuous optimization and adjustment of the reinforcement learning strategy model. In this way, the present application can significantly improve the overall performance and resource utilization efficiency of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 This is a flow chart of the first embodiment of the automatic networking method for spatial computing of the present application;

[0042] Figure 2 This is a flow chart of the second embodiment of the automatic networking method for spatial computing of the present application;

[0043] Figure 3 This is another flowchart of the second embodiment of the automatic networking method for spatial computing of the present application;

[0044] Figure 4 This is a schematic diagram of the module structure of the automatic networking device for spatial computing according to an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the automatic networking method for spatial computing in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a terminal system, etc. The following takes the system as an example to illustrate this embodiment and the following embodiments.

[0050] Based on this, this embodiment provides a spatial computing automatic networking method, referring to Figure 1 , Figure 1 This is a flow chart of the automatic networking method for spatial computing of the present application. The automatic networking method for spatial computing includes steps S10 to S40:

[0051] Step S10, constructing a super object model, performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices;

[0052] It should be noted that the Super Object Model is an abstract model that integrates the hardware specifications, dynamic status, and spatial location information of all preset devices in the network. This model provides basic data support for automatic networking, so that the construction and management of the network can be optimized based on the capabilities of these devices. Hardware specifications refer to the physical configuration of the device, including the number of CPU cores, memory size, storage capacity, etc., which are the basic parameters of device performance. Dynamic status refers to the state of the device at runtime, such as current load, available resources, etc. These states change over time and are crucial for real-time adjustment of the network. Spatial location refers to the exact location of the device in the physical space, including geographical location, physical height, etc., which has a direct impact on the geographical layout and signal coverage of the network. The initial network topology is the network topology obtained based on the Super Object Model using a preset clustering algorithm. It defines the initial connection method of devices in the network.

[0053] First, the hardware specifications, dynamic status, and spatial location information of all preset devices need to be collected. This information is integrated into an abstract model, the super-object model. This model contains not only static information about the device (such as hardware specifications), but also dynamic information (such as current load and available resources), as well as spatial information such as the device's geographic location. Based on the device information in the super-object model, a preset clustering algorithm (such as Kmeans clustering) is used to generate the initial network topology.

[0054] Step S20, obtaining the current device state of the device corresponding to the initial network topology structure, inputting the current device state into a pre-trained reinforcement learning strategy model, and outputting a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure;

[0055] It should be noted that the current device state refers to the real-time state information of the device at a certain moment, including but not limited to the actual usage of its hardware specifications, changes in dynamic states, etc. The reinforcement learning strategy model is a machine learning model that makes decisions by continuously trying and learning in a way that maximizes a certain reward signal. In this embodiment, it can select the optimal network topology structure based on the current device state of the device, and is used to optimize the network configuration based on the device state and the network topology structure. The first strategy is a network configuration scheme output by the reinforcement learning strategy model based on the current device state and the initial network topology structure, which is used to guide the update of the network topology structure.

[0056] Get the current device status of the devices corresponding to the initial network topology. This status information includes the real-time operating data of the devices, such as power consumption, temperature, etc., as well as their relative positions in the network. This information will be input into the pre-trained reinforcement learning policy model. The reinforcement learning policy model outputs the first policy based on the input device status information and the initial network topology. This policy is derived by the model through continuous learning and trial, and is designed to optimize network configuration and improve network performance.

[0057] Step S30, updating the initial network topology structure according to the first strategy to obtain a first network topology structure;

[0058] Step S40, determining the network performance of the first network topology structure, and if the network performance reaches a preset performance threshold, determining the first network topology structure as the target network topology structure.

[0059] It should be noted that the first network topology is the network topology updated according to the reinforcement learning strategy model and state vector. It is an intermediate state in the network optimization process. Network performance refers to the various network performances exhibited by the network system under specific conditions (such as given state vectors and action vectors), such as latency, throughput, resource utilization, etc. These network performances are key parameters for measuring network efficiency and effectiveness. The performance threshold is a preset performance standard used to determine whether the network topology meets the performance requirements. If the network performance of the first network topology reaches this threshold, it will be determined as the final target network topology.

[0060] The initial network topology is updated using the generated first strategy. This process involves adjusting the connection relationship or resource allocation strategy between devices according to the action vector selected by the strategy model to generate the first network topology. Next, the network performance of the first network topology is determined. This step is accomplished by measuring key performance indicators such as network throughput and latency. If the network performance reaches the preset performance threshold, the first network topology is determined as the target network topology. This means that the network configuration has been optimized to a state that meets the requirements and can be put into use. If not, it may be necessary to further adjust the first strategy, update the weight parameters of the first strategy, and re-update and evaluate the network topology.

[0061] This embodiment constructs a super object model, wherein the super object model includes the hardware specifications, dynamic states, and spatial locations of all preset devices. By abstracting and integrating device information, this embodiment forms a virtual resource pool to provide basic data support for subsequent network optimization and decision-making; cluster analysis is performed on at least two devices based on the super object model to obtain an initial network topology structure to optimize network performance and resource utilization; the current device state of the device corresponding to the initial network topology structure is obtained, and the current device state is input into a pre-trained reinforcement learning strategy model, and the first strategy is output to enable it to select the optimal action according to the current network state to maximize long-term rewards, such as improving network performance and reducing resource consumption, etc., wherein the reinforcement learning strategy model is a strategy model constructed and trained based on the super object model and the initial network topology structure; the initial network topology structure is updated according to the first strategy to obtain a first network topology structure, and the network performance of the first network topology structure is determined. If the network performance reaches a preset performance threshold, the first network topology structure is determined as the target network topology structure, and the network topology structure that meets the network performance requirements is gradually found through continuous optimization and adjustment of the reinforcement learning strategy model. In this way, the present application can significantly improve the overall performance and resource utilization efficiency of the network.

[0062] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction, and will not be repeated in the following. Figure 2 The step S10 further includes steps A10 to A40:

[0063] Step A10, generating an attribute vector according to the hardware specifications and dynamic state of the device;

[0064] It should be noted that the attribute vector is a vector composed of the device's hardware specifications (such as the number of CPU cores, memory size, network bandwidth, etc.) and dynamic status (such as current load, available resources, etc.), which is used to describe the device's hardware resources and dynamic status.

[0065] Collect the hardware specifications and dynamic status information of the device. This information includes hardware specifications such as the number of CPU cores, memory size, network bandwidth, and dynamic status such as current load and available resources. This information is standardized and combined into a vector, namely the attribute vector. This vector can fully describe the hardware resources and dynamic status of the device.

[0066] Step A20, generating a three-dimensional vector according to the spatial position information of the device;

[0067] It should be noted that a three-dimensional vector is a vector composed of the spatial position information of a device (such as geographical location, physical space position, etc.), usually including three coordinate values ​​of x, y, and z, which are used to represent the position of the device in space.

[0068] Collect the spatial location information of the device, including the geographic location (such as longitude and latitude) and physical location (such as floor, room number, etc.). Encode this information into a three-dimensional vector, where the x, y, and z coordinates represent the different dimensional locations of the device in space. This three-dimensional vector can accurately represent the location information of the device in space.

[0069] Step A30, defining a metadata field for each device, and generating a metadata vector based on the metadata field;

[0070] It should be noted that the metadata vector is a vector composed of additional information fields (such as device type, manufacturer, firmware version, etc.) defined for each device, and is used to extend the description information of the device.

[0071] Define additional information fields for each device, such as device type, manufacturer, firmware version, etc. Combine this information into a vector, the metadata vector. This vector can extend the device description information and provide additional device characteristics.

[0072] Step A40, constructing a super object model according to the attribute vector, the three-dimensional vector, the metadata vector, and a preset time series, wherein the time series represents the data differences generated by the attribute vector, the three-dimensional vector, and the metadata vector respectively changing over time.

[0073] It should be noted that the time series is a data series that represents the changes in device status over time, and is used to record the historical changes and trends of device status.

[0074] Record the data of device status changes over time, including historical load, resource utilization, etc. Organize this data into time series to represent the historical changes and trends of device status. Time series can provide dynamic information about device status, which is helpful for subsequent network optimization and decision-making. The super object model forms a virtual resource pool by integrating attribute vectors, time series, three-dimensional vectors and metadata vectors, which can comprehensively describe the hardware resources, dynamic status, spatial location and additional information of the device. In the subsequent network optimization and decision-making process, the super object model will serve as an important data foundation to support the implementation of automatic networking methods.

[0075] Exemplarily, the hardware specifications of the device (such as the number of CPU cores, memory size, network bandwidth, etc.), the dynamic state of the device (such as current load, available resources, etc.), and the spatial attributes of the device (such as geographic location, physical space location, etc.) can be used as input dependencies. The hardware specifications and dynamic state of the acquisition device are standardized and described as an attribute vector D = [d_1, d_2, ..., d_n], where d_i represents the value of the i-th attribute. The spatial attributes of the device are encoded as a three-dimensional vector S = [s_x, s_y, s_z]. Additional metadata fields are defined for each device, such as device type, manufacturer, firmware version, etc., to form a metadata vector M = [m_1, m_2, ..., m_k]. Time series data T = [t_1, t_2, ..., t_m] is used to represent the change of device status over time, and the device status is monitored in real time, and the data in the super object model is updated regularly. Finally, the attribute vector, three-dimensional vector, metadata vector and time series are combined into a super object model O = {D, S, M, T}, which is used as the basis for subsequent network optimization and decision-making.

[0076] This embodiment successfully builds a super object model, providing comprehensive data support for subsequent network optimization and decision-making. This process reflects the complete process from device information collection to model construction, ensuring the accuracy and practicality of the super object model.

[0077] In one possible implementation, refer to Figure 3 The step S10 further includes steps B10 to B20:

[0078] Step B10: based on the super object model, using a preset clustering algorithm to group at least two devices to obtain at least one subnet, wherein the clustering algorithm groups devices with similar functions or close spatial locations into the same subnet according to the super object model;

[0079] It should be noted that a clustering algorithm is an algorithm that divides a data set into multiple clusters or groups, so that the data points within the same cluster are similar to each other, while the data points between different clusters are quite different. In this step, it is used to group devices into multiple subnets. In a network, a subnet is a logical grouping of a part of devices that communicate within the subnet through specific connection relationships.

[0080] First, according to the device information in the super-object model (including hardware specifications, dynamic status, spatial attributes, etc.), a clustering algorithm (such as Kmeans clustering) is used to group the devices. The clustering algorithm divides the devices into different clusters based on the similarity between devices (such as functional attributes, spatial location, etc.), and each cluster represents a subnet. The clustering algorithm calculates the distance or similarity between devices and divides similar devices into the same subnet for subsequent network optimization and management. After the clustering algorithm is processed, multiple subnets are obtained, each of which contains a group of similar devices.

[0081] Step B20, for each subnet, determining a first connection relationship between all devices belonging to the subnet, and optimizing the structure of the subnet according to the first connection relationship, wherein the initial network topology structure includes each subnet after structural optimization.

[0082] Within each subnet, a preset connection algorithm (such as the shortest path algorithm, spanning tree algorithm, etc.) is used to generate the connection relationship between devices. These algorithms take into account factors such as the hardware resources, dynamic state, and spatial location of the device to determine the optimal connection path and connection method. The connection algorithm calculates the connection cost and benefit between devices and selects the optimal connection path and connection method to ensure the minimum delay and maximum resource complementarity of the device connection within the subnet. Within each subnet, the connection relationship between devices is generated to form a preliminary network structure within the subnet. All subnets obtained (including the connection relationship within the subnet) are gathered together to form the initial network topology. This topology describes the connection relationship between devices and the layout between subnets.

[0083] This embodiment uses a clustering algorithm to group devices based on the super object model to form multiple subnets; then uses a connection algorithm to generate connection relationships between devices within each subnet; and finally brings all subnets together to form an initial network topology. This structure provides a basis for subsequent network optimization and decision-making.

[0084] Based on the first or second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first or second embodiment can be referred to the above introduction, and will not be described in detail later. Step S30 also includes steps C10 to C20:

[0085] Step C10, if the initial network topology structure has multiple subnets, determine whether there is a policy conflict between the first policies corresponding to each subnet;

[0086] It should be noted that, in this embodiment, each subnet will generate a first policy corresponding to itself, and policy conflicts may occur between the first policies of different subnets. Policy conflict occurs when two or more policies are applied at the same time, but their goals, methods or results contradict each other, resulting in a situation where they cannot be satisfied or achieved at the same time.

[0087] First, determine whether there are multiple subnets in the initial network topology. In large networks, for the convenience of management and optimization, the network is usually divided into multiple subnets. If there are indeed multiple subnets in the initial network topology, then it is necessary to perform conflict detection on the first policy corresponding to each subnet. The specific process of conflict detection is to compare the first policy of each subnet to see if there are any contradictions or situations that cannot be implemented at the same time. For example, if the first policies of two subnets both require configuration of a specific device, but the configuration parameters conflict with each other, then the two policies are in conflict.

[0088] Furthermore, when there are multiple subnets in the initial network topology, a multi-agent collaboration mechanism can be introduced. An agent is set up in each subnet, and different agents are responsible for the management of different subnets. These agents have the ability to autonomously perceive the environment, process data, and make decisions. They can share status information and strategies, and are used to collect subnet status information in real time or regularly and send it to the central node (such as the central control unit). The central node integrates all subnet status information and determines whether there is a policy conflict between different subnets.

[0089] Step C20: If there is no policy conflict between the first policies corresponding to each subnet, then for each subnet, the network topology structure of the subnet is updated according to the first policy corresponding to each subnet to obtain a first network topology structure.

[0090] If it is found through testing that there is no conflict between the first strategies corresponding to each subnet, then the network topology of each subnet can be updated according to its corresponding first strategy. The updating process may include adjusting the connection relationship of the device, changing the bandwidth of the link, optimizing the configuration parameters of the device, etc. These operations are performed according to the optimization suggestions in the first strategy, aiming to improve the performance, stability and security of the network.

[0091] This embodiment effectively solves the problem of possible policy conflicts between multiple subnets in the initial network topology by introducing a policy conflict detection mechanism and a multi-agent collaboration mechanism. When no policy conflict is detected, the network topology of each subnet can be accurately updated according to the first policy of each subnet, thereby optimizing network performance, enhancing stability and security. This process not only avoids contradictions between policies, but also ensures the rationality and efficiency of network configuration, providing strong support for the optimization management of large networks. In a feasible implementation, step C10 may include steps D10 to D30:

[0092] Step D10: if there are at least two subnets corresponding to the first policies that have policy conflicts, then the subnets with policy conflicts are determined as conflicting subnets;

[0093] It should be noted that a subnet with conflicting policies is called a conflicting subnet. Among multiple subnets, if the first policies of at least two subnets conflict, these subnets are conflicting subnets.

[0094] Step D20, obtaining a global policy model sent by a preset central control unit, wherein the central control unit determines a global network state according to the subnet states of all subnets, and constructs a global policy model according to the global network state;

[0095] It should be noted that the central control unit is a device or system component responsible for monitoring and managing the entire network system. It can collect subnet status information of all subnets, determine the global network status based on it, and build a global policy model to optimize the performance and stability of the entire network.

[0096] Each subnet obtains (if a multi-agent collaboration mechanism is introduced, the subnet’s agents can be used to obtain) a global policy model sent by the preset central control unit. This model is constructed by the central control unit based on the global network state determined by the subnet state information of all subnets. It takes into account the needs and states of all subnets and aims to achieve the global optimal network performance.

[0097] Step D30, determining a second decision of the conflicting subnet according to the global policy model, and updating the network topology structure of the conflicting subnet according to the second decision to obtain an updated conflicting subnet;

[0098] It should be noted that the global policy model is a policy model built based on the global network status, which is used to guide the configuration and optimization of the entire network. It takes into account the status and needs of all subnets and aims to achieve the global optimal network performance. The second decision is a new network configuration or optimization plan made according to the global policy model for the conflicting subnets. This decision aims to resolve policy conflicts, enable the conflicting subnets to coexist harmoniously with other subnets, and optimize the performance of the entire network.

[0099] The second decision of the conflicting subnet is determined according to the global policy model. This decision is made specifically for the conflicting subnet, aiming to resolve the policy conflict and maintain a consistent network configuration with other subnets. This decision may include adjusting the connection relationship of the device, changing the bandwidth of the link, optimizing the configuration parameters of the device, etc. Then, the network topology of the conflicting subnet is updated according to the second decision to obtain the updated conflicting subnet. This subnet has resolved the policy conflict and maintained a harmonious network configuration with other subnets.

[0100] Step D40, for the subnets without policy conflicts, updating the network topology structure according to the first policy corresponding to the subnets without policy conflicts, to obtain a second subnet;

[0101] It should be noted that the second subnet is a subnet obtained by updating the network topology structure according to the corresponding first policy for the subnet without policy conflict. These subnets maintain the original network configuration and form a new network topology structure together with other subnets (including the updated conflicting subnet).

[0102] Step D50: determining the first network topology structure according to the updated conflicting subnet and / or the second subnet.

[0103] At the same time, for the subnets without policy conflicts, the network topology is updated according to the corresponding first policy. Finally, the first network topology is determined according to the updated conflicting subnets and / or the second subnets. This new network topology is the result of the application of the global policy model, which takes into account the needs and status of all subnets and achieves the global optimal network performance.

[0104] This embodiment effectively solves the problem of policy conflicts between subnets by introducing a global policy model. When a policy conflict is detected, the global policy model constructed by the central control unit is used to make a second decision for the conflicting subnet, and its network topology is updated accordingly to ensure harmonious coexistence between subnets. At the same time, for subnets without policy conflicts, they are updated according to their first policy, and finally the globally optimal first network topology is obtained, which improves the performance and stability of the network.

[0105] In a feasible implementation, the spatial computing automatic networking method may include steps E10 to E30:

[0106] Step E10, defining the hyper-object model at a preset initial moment as a state vector, defining an action vector according to an adjustment strategy between devices, and defining a strategy model based on a mapping relationship between the state vector and the action vector;

[0107] It should be noted that the state vector refers to a set of parameters that describe the current state of each device in the network environment. In this embodiment, it is the current state of each device at a preset initial moment, including but not limited to the device's hardware resources (such as CPU, memory usage), software function status, dynamic state (such as current load, available resources) and spatial attributes (such as geographic location, physical height, etc.). The initial moment can be a moment in the time series, not just the moment when the super object model is created. The action vector refers to a set of operations or decisions that the reinforcement learning model can perform in a given state, such as adjusting the connection relationship between devices, reallocating resources, etc. The selection of action vectors aims to optimize network performance, such as reducing latency, improving throughput or resource utilization. The policy model is a model defined based on the mapping relationship between the state vector and the action vector, which is used to decide the adjustment strategy of the network. It is trained through a learning algorithm to optimize network performance.

[0108] The super object model at the initial moment is defined as a state vector. This state vector contains all the key information of the super object model at the initial moment, such as the state of the device, attribute values, etc. At the same time, action vectors are defined according to the adjustment strategy between devices. These action vectors represent a series of actions or a collection of operations that the device can take. Next, a policy model is defined based on the mapping relationship between the state vector and the action vector. This policy model describes what actions the device should take in different states to achieve the best network performance. In order to evaluate the effects of these actions, it is necessary to determine the network performance of the state vector and the action vector. This network performance can be obtained by measuring key indicators such as the network's transmission efficiency, delay, and packet loss rate.

[0109] Step E20, determining the network performance under the state vector and the action vector, and defining a reward function according to the network performance, wherein the reward function is used to evaluate the network performance of the action vector under the state vector;

[0110] It should be noted that network performance refers to key indicators such as transmission efficiency, delay, and packet loss rate of the network at a certain moment. These indicators are used to evaluate the operating status of the network and the performance of the device. The reward function is defined based on network performance and is used to evaluate the network performance of the action vector under the state vector. It gives the reward value that can be obtained by taking different actions in different states, which is used to guide the learning and optimization of the policy model.

[0111] Define a reward function based on network performance. This reward function gives the reward value that can be obtained by taking different actions in different states. The larger the value of the reward function, the better the network performance of the action in this state. This reward function is used to guide the learning and optimization of the policy model.

[0112] Step E30, training the strategy model with a learning algorithm according to the reward function to obtain a reinforcement learning strategy model.

[0113] The policy model is trained with a learning algorithm based on the reward function to obtain a reinforcement learning policy model. During the training of the learning algorithm, the policy model will continuously try different action vectors and adjust its decisions based on the feedback of the reward function. Through repeated training and learning, the policy model can gradually find the optimal action vector to maximize network performance. The resulting reinforcement learning policy model can select the optimal action vector based on the current state vector, thereby optimizing the state and performance of the entire network.

[0114] Exemplarily, when training a reinforcement learning policy model, the state space is first defined, that is, the current state of each device (the current state of the super object model) is concretized into a state vector S_t containing a time step t, S_t = [D_t, S_t, M_t, T_t] t represents the time step. Subsequently, the action space A is defined, which contains a variety of possible actions, such as adjusting the connection relationship between devices, optimizing resource allocation strategies, etc., specifically expressed as A = {a_1, a_2, ..., a_p}. In order to evaluate the effects of these actions, a reward function R (S_t, A_t) is designed, which is calculated based on the key performance indicators of the network (such as latency, throughput, and resource utilization). Next, a reinforcement learning algorithm such as a deep Q network or a policy gradient is used to gradually optimize the policy model through multiple rounds of iterative training, so that it can select the optimal action according to different states. Finally, a trained reinforcement learning policy model π_θ is output, which can make dynamic decisions in a real-time network environment to optimize network performance and resource utilization.

[0115] Exemplarily, when applying the reinforcement learning strategy model to generate the network topology, according to the state vector S_t, the reinforcement learning strategy model π_θ will dynamically select the optimal action A_t, which belongs to the action set A, and is intended to adjust the connection relationship between devices or reallocate resources to optimize the network. Subsequently, the action A_t is executed to update the current network topology G_t (t represents the time step), thereby generating a new network structure G_{t+1}. Next, based on the performance indicators of the new network structure G_{t+1} (such as delay, throughput, resource utilization, etc.), the reward value R(S_t, A_t) is calculated. This reward value reflects the impact of action A_t on network performance. Based on the obtained reward value, the parameters of the strategy model π_θ are updated to make it more accurate and efficient in future decisions. This process of dynamic decision-making, action implementation, reward feedback and strategy model update will be repeated until the network topology and performance indicators reach the preset optimization conditions. Finally, the optimized network topology structure G is output, which not only meets the current device status, but also complies with business requirements and improves network performance.

[0116] This embodiment evaluates network performance through state vectors and action vectors, and then designs a reward function to train and optimize the strategy model, and finally obtains a reinforcement learning strategy model that can intelligently adjust the network structure and resource allocation strategy. The reinforcement learning strategy model can be applied to actual network systems and intelligently selects the optimal action vector according to the real-time state vector to optimize network performance.

[0117] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the spatial computing automatic networking method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0118] This application also provides a spatial computing automatic networking device, please refer to Figure 4 , the spatial computing automatic networking device includes:

[0119] A super object model generation module 10 is used to construct a super object model, and to perform cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices;

[0120] The strategy generation module 20 obtains the current device state of the device corresponding to the initial network topology structure, inputs the current device state into a pre-trained reinforcement learning strategy model, and outputs a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure;

[0121] A network topology generation module 30 updates the initial network topology structure according to the first strategy to obtain a first network topology structure;

[0122] The performance detection module 40 determines the network performance of the first network topology structure, and if the network performance reaches a preset performance threshold, determines the first network topology structure as the target network topology structure.

[0123] The spatial computing automatic networking device provided by the present application adopts the spatial computing automatic networking method in the above embodiment, which can solve the technical problem of low network performance in the dynamic networking environment of the Internet of Things devices. Compared with the prior art, the beneficial effects of the spatial computing automatic networking device provided by the present application are the same as the beneficial effects of the spatial computing automatic networking method provided by the above embodiment, and the other technical features in the spatial computing automatic networking device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0124] The present application provides a spatial computing automatic networking device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the spatial computing automatic networking method in the above-mentioned embodiment one.

[0125] Reference below Figure 5 , which shows a schematic diagram of the structure of a spatial computing automatic networking device suitable for implementing the embodiment of the present application. The spatial computing automatic networking device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The spatial computing automatic networking device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0126] like Figure 5As shown, the spatial computing automatic networking device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the spatial computing automatic networking device are also stored in RAM1004. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the spatial computing automatic networking device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a spatial computing automatic networking device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0127] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0128] The spatial computing automatic networking device provided by the present application adopts the spatial computing automatic networking method in the above embodiment, which can solve the technical problem of low network performance in the dynamic networking environment of the Internet of Things devices. Compared with the prior art, the beneficial effects of the spatial computing automatic networking device provided by the present application are the same as the beneficial effects of the spatial computing automatic networking method provided by the above embodiment, and the other technical features in the spatial computing automatic networking device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0130] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0131] The present application provides a medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the spatial computing automatic networking method in the above-mentioned embodiment.

[0132] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0133] The above-mentioned computer-readable storage medium may be included in the spatial computing automatic networking device; or it may exist independently without being assembled into the spatial computing automatic networking device.

[0134] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the spatial computing automatic networking device, the spatial computing automatic networking device:

[0135] Constructing a super object model, and performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices;

[0136] Obtaining the current device state of the device corresponding to the initial network topology structure, inputting the current device state into a pre-trained reinforcement learning strategy model, and outputting a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure;

[0137] The initial network topology structure is updated according to the first strategy to obtain a first network topology structure;

[0138] The network performance of the first network topology is determined, and if the network performance reaches a preset performance threshold, the first network topology is determined as the target network topology.

[0139] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0142] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned spatial computing automatic networking method, and can solve the technical problem of low network performance in the dynamic networking environment of IoT devices. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the spatial computing automatic networking method provided by the above-mentioned embodiment, and will not be repeated here.

[0143] The present application also provides a product, which is a computer program product, including a computer program, which implements the steps of the above-mentioned spatial computing automatic networking method when executed by a processor.

[0144] The computer program product provided by this application can solve the technical problem of low network performance in a dynamic networking environment of IoT devices. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the spatial computing automatic networking method provided by the above embodiment, and will not be repeated here.

[0145] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A spatial computing automatic networking method, characterized in that: The spatial computing automatic networking method comprises: Constructing a super object model, and performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices; Acquire the current device state of the device corresponding to the initial network topology structure, input the current device state into a pre-trained reinforcement learning strategy model, and output a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure; Updating the initial network topology structure according to the first strategy to obtain a first network topology structure; The network performance of the first network topology is determined, and if the network performance reaches a preset performance threshold, the first network topology is determined to be a target network topology.

2. The automatic spatial computing networking method according to claim 1, characterized in that: The steps of constructing the super object model include: generating an attribute vector according to the hardware specifications and dynamic state of the device; generating a three-dimensional vector according to the spatial position information of the device; defining a metadata field for each of the devices, and generating a metadata vector based on the metadata field; A super object model is constructed according to the attribute vector, the three-dimensional vector, the metadata vector, and a preset time series, wherein the time series represents data differences generated by the attribute vector, the three-dimensional vector, and the metadata vector respectively changing over time.

3. The automatic spatial computing networking method according to claim 1, characterized in that: The step of performing cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure includes: Based on the super object model, the at least two devices are grouped using a preset clustering algorithm to obtain at least one subnet, wherein the clustering algorithm groups devices with similar functions or close spatial locations into the same subnet according to the super object model; For each subnet, a first connection relationship between all devices belonging to the subnet is determined, and the subnet is structurally optimized according to the first connection relationship, wherein the initial network topology structure includes each subnet after structural optimization.

4. The automatic spatial computing networking method according to claim 1, characterized in that: The step of updating the initial network topology structure according to the first strategy to obtain a first network topology structure includes: If the initial network topology structure has multiple subnets, determining whether there is a policy conflict between the first policies corresponding to each subnet; If there is no policy conflict between the first policies corresponding to each subnet, then for each subnet, the network topology structure of the subnet is updated according to the first policy corresponding to each subnet to obtain a first network topology structure.

5. The automatic spatial computing networking method according to claim 4, characterized in that: After the step of determining whether there is a policy conflict between the first policies corresponding to each subnet, the method further includes: If there are at least two subnets corresponding to the first policies that have policy conflicts, the subnets with policy conflicts are determined as conflicting subnets; Obtaining a global policy model sent by a preset central control unit, wherein the central control unit determines a global network state according to the subnet states of all subnets, and constructs a global policy model according to the global network state; Determine a second decision for the conflicting subnet according to the global policy model, and update the network topology structure of the conflicting subnet according to the second decision to obtain an updated conflicting subnet; For the subnets without policy conflicts, the network topology structure is updated according to the first policy corresponding to the subnets without policy conflicts to obtain the second subnets; A first network topology is determined according to the updated conflicting subnet and / or the second subnet.

6. The automatic spatial computing networking method according to any one of claims 1 to 5, characterized in that: The spatial computing automatic networking method also includes: The super object model at a preset initial moment is defined as a state vector, an action vector is defined according to the adjustment strategy between the devices, and a strategy model is defined based on the mapping relationship between the state vector and the action vector; Determine the network performance under the state vector and the action vector, and define a reward function according to the network performance, wherein the reward function is used to evaluate the network performance of the action vector under the state vector; The strategy model is trained by a learning algorithm according to the reward function to obtain a reinforcement learning strategy model.

7. A spatial computing automatic networking device, characterized in that: The spatial computing automatic networking device comprises: A super object model generation module, which constructs a super object model, and performs cluster analysis on at least two devices based on the super object model to obtain an initial network topology structure, wherein the super object model includes hardware specifications, dynamic states, and spatial locations of all preset devices; a strategy generation module, which obtains the current device state of the device corresponding to the initial network topology structure, inputs the current device state into a pre-trained reinforcement learning strategy model, and outputs a first strategy, wherein the reinforcement learning strategy model is a strategy model constructed and trained according to the super object model and the initial network topology structure; A network topology generation module updates the initial network topology structure according to the first strategy to obtain a first network topology structure; The performance detection module determines the network performance of the first network topology structure, and if the network performance reaches a preset performance threshold, determines that the first network topology structure is a target network topology structure.

8. A spatial computing automatic networking device, characterized in that: The spatial computing automatic networking device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the spatial computing automatic networking method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the spatial computing automatic networking method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the spatial computing automatic networking method according to any one of claims 1 to 6 are implemented.

Images