Method, device, equipment, medium and product for assembling distributed reasoning cluster

Abstract

The application relates to the technical field of computers and discloses a method, device, equipment, medium and product for establishing a distributed reasoning cluster. The method comprises the following steps: obtaining a first external network connectivity state of a first terminal device; obtaining a link layer broadcast message of a second terminal device based on a message listening thread of the first terminal device; analyzing the link layer broadcast message to obtain a second external network connectivity state of the second terminal device; determining a master device and a slave device from the first terminal device and the second terminal device based on the first external network connectivity state and the second external network connectivity state; and in response to the master device and the slave device passing a connectivity verification, establishing the distributed reasoning cluster on the master device and the slave device. Through implementation of the application, zero-configuration automatic networking between multiple terminal devices is realized, the deployment operation threshold of the distributed reasoning cluster between the multiple terminal devices is reduced, and the deployment efficiency of the distributed reasoning cluster is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to methods, apparatus, equipment, media, and products for building distributed inference clusters. Background Technology

[0002] As large-scale model development toolchains mature, the development threshold for intelligent agents has been significantly lowered, and intelligent agents in various vertical fields have been deployed in batches, making integrated artificial intelligence (AI) boxes the mainstream terminal devices.

[0003] Currently, deploying inference clusters across multiple terminal devices integrated with AI smart boxes primarily relies on manual configuration of each device and pre-setting of master-slave roles. This deployment process is complex and has a high barrier to entry, making it difficult to quickly build a cluster without professional maintenance personnel. Consequently, the deployment efficiency of distributed inference clusters among multiple AI terminal devices is low, failing to achieve plug-and-play networking and thus failing to meet the needs of rapid network inference at the edge. Summary of the Invention

[0004] This invention provides a method, apparatus, device, medium, and product for building a distributed inference cluster, in order to solve the problem of low deployment efficiency of distributed inference clusters.

[0005] In a first aspect, the present invention provides a method for constructing a distributed inference cluster, comprising: obtaining a first external network connectivity status of a first terminal device; obtaining a link layer broadcast message of a second terminal device based on a message listening thread of the first terminal device, wherein the second terminal device is the peer device of the first terminal device; exchanging information between the first terminal device and the second terminal device based on the link layer broadcast message; parsing the link layer broadcast message to obtain device information of the second terminal device, wherein the device information includes a second external network connectivity status and a second device identifier; determining a master device and a slave device from the first terminal device and the second terminal device based on the first external network connectivity status and the second external network connectivity status; and constructing a distributed inference cluster on the master device and the slave device in response to the master device and the slave device passing connectivity verification.

[0006] The distributed inference cluster construction method provided in this invention involves the first and second terminal devices exchanging information via link-layer broadcast messages. Since there is no central information interaction, the external network connectivity status of both parties can be automatically collected without the need for a DHCP server. Based on the external network connectivity status, the master-slave role election is automatically completed between the first and second terminal devices, eliminating the need for manual configuration. After the master-slave connectivity verification is successful, a distributed inference cluster is automatically built on the master and slave devices. This achieves zero-configuration automatic networking and inference between multiple terminal devices, enabling rapid networking in edge inference scenarios such as those without maintenance. It significantly reduces the deployment threshold of distributed inference clusters between multiple terminal devices, improves deployment efficiency, and facilitates plug-and-play networking.

[0007] In one optional implementation, parsing the link layer broadcast message to obtain device information of the second terminal device includes: responding to the link layer broadcast message being a User Datagram Protocol (UDP) message, parsing the UDP message based on the UDP to obtain the second external network connectivity status of the second terminal device; or, responding to the link layer broadcast message being a Dynamic Host Configuration Protocol (DHCP) discovery message, parsing the additional information field of the DHCP discovery message based on the DHCP to obtain the second external network connectivity status corresponding to the additional information field; or, responding to the link layer broadcast message being a Link Layer Discovery Protocol (LINK) message, parsing the type-length-value field of the LINK message based on the LINK to obtain the second external network connectivity status of the second terminal device.

[0008] The distributed inference cluster construction method provided in this invention is compatible with three types of link-layer broadcast messages: User Datagram Protocol (UDP), Dynamic Host Configuration Protocol (DHCP), and Link Layer Discovery Protocol (LLP). It does not rely on proprietary protocols, thus adapting to different first-terminal devices' information reporting interaction methods and improving versatility. Furthermore, relying solely on Layer 2 link-layer broadcast messages for information exchange eliminates the need for pre-allocation of Layer 3 network addresses, enabling rapid acquisition of peer device information even without a central network.

[0009] In an optional implementation, the method further includes: parsing the link layer broadcast message to obtain the device name of the second terminal device, the device information including the device name; in response to the device name passing the validity check, storing the device information of the second terminal device in the neighbor node mapping table corresponding to the first terminal device; in response to the device name failing the validity check, discarding the link layer broadcast message of the second terminal device.

[0010] The distributed inference cluster construction method provided in this invention parses device names from link-layer broadcast messages to perform validity checks, filtering out illegal and abnormal link-layer broadcast messages. This prevents malicious messages from interfering with the network construction process of the distributed inference cluster, thus improving network stability. By storing the device information that has passed the validity check in a neighbor node mapping table, it is easy to directly look up and reuse the table later without repeated packet capture and parsing, further improving network response speed.

[0011] In one optional implementation, determining a master device and a slave device from a first terminal device and a second terminal device based on a first external network connectivity status and a second external network connectivity status includes: determining whether the first external network connectivity status and the second external network connectivity status are consistent; if the first external network connectivity status and the second external network connectivity status are consistent, then obtaining a first device identifier corresponding to the first terminal device and a second device identifier corresponding to the second terminal device; converting the first device identifier and the second device identifier into a first value and a second value of preset bytes; determining a master device and a slave device from the first terminal device and the second terminal device based on the comparison result of the first value and the second value; if the first external network connectivity status indicates that the first terminal device has external network access capability, while the second external network connectivity status indicates that the second terminal device does not have external network access capability, then the first terminal device is determined as the master device and the second terminal device is determined as the slave device; if the first external network connectivity status indicates that the first terminal device does not have external network access capability, while the second external network connectivity status indicates that the second terminal device has external network access capability, then the second terminal device is determined as the master device and the first terminal device is determined as the slave device.

[0012] The distributed inference cluster construction method provided in this invention, when the external network connectivity status of the first terminal device and the second terminal device is consistent, converts the device identifiers of the first terminal device and the second terminal device into preset byte values ​​for comparison. The determination rules are unified, the calculation logic is simple, and it is easy to implement at the device level. Hierarchical election is performed using external network connectivity status as the first priority and device identifier value as the second priority, ensuring that the determination of master and slave devices has a unified and quantifiable standard, and avoiding master-slave device election conflicts.

[0013] In one optional implementation, connectivity verification of the master device and the slave device includes: detecting whether the master device is configured with a first master address within the target network segment, and whether the slave device is configured with a first slave address within the target network segment; if the master device is configured with a first master address and the slave device is configured with a first slave address, then connectivity verification of the master device and the slave device is performed based on the first master address and the first slave address; if the master device is not configured with a first master address, and / or the slave device is not configured with a first slave address, then a preset second master address is configured for the master device, and a preset second slave address is configured for the slave device; connectivity verification of the master device and the slave device is performed based on the second master address and the second slave address.

[0014] The distributed inference cluster construction method provided in this invention verifies connectivity based on the configured network addresses when the first and second terminal devices are detected to have network addresses in the target network segment. This eliminates the need for redundant address allocation, reuses existing network configurations, and shortens network setup time. Bidirectional connectivity verification based on network addresses in the same network segment accurately identifies network faults such as port, link, and network segment disconnections, ensuring reliable communication links for the subsequently constructed distributed inference cluster. When the first and / or second terminal devices do not have network addresses in the target network segment, network addresses are automatically allocated, achieving self-allocation of addresses without a DHCP server and truly achieving zero-configuration of network addresses. Furthermore, by automatically allocating network addresses in the target network segment, the problem of device inability to interconnect due to network address mismatches can be avoided, improving the compatibility of self-organizing networks.

[0015] In an optional implementation, the method further includes: in response to the existence of multiple slave devices, obtaining a third device identifier corresponding to each slave device; converting each third device identifier into a third value of preset bytes; and configuring a preset second slave address for each slave device in sequence according to the sorting result of the third value.

[0016] The distributed inference cluster construction method provided in this invention, for multiple slave devices, converts the third device identifier corresponding to each slave device into a third value of preset bytes, and allocates the network addresses of each slave device in an orderly manner according to the third values ​​to avoid network address allocation conflicts. Moreover, by allocating network addresses to each slave device according to the sorting result of the third values, the allocation result of the slave device network addresses is guaranteed to be consistent in each network formation, which facilitates device management, device location, and operation and maintenance troubleshooting in the distributed inference cluster.

[0017] In an optional implementation, the method further includes: in response to the master device and slave device failing connectivity verification, obtaining the number of connectivity verification failures; in response to the number of connectivity verification failures not exceeding a preset value, restarting connectivity verification of the master device and slave device at a preset interval; in response to the number of connectivity verification failures exceeding a preset value, resetting the network addresses of the master device and slave device, and performing connectivity verification based on the reset network addresses.

[0018] The distributed inference cluster construction method provided in this invention employs an interval retry mechanism after connectivity verification failures. This allows for automatic recovery from occasional fault scenarios such as temporary network jitter and packet loss, without requiring manual restart of the network setup process. After the number of failures reaches a preset value, the network address is automatically reset and connectivity verification is re-executed, avoiding persistent network faults such as network segment conflicts and address occupancy. This enhances self-healing capabilities and reduces on-site manual intervention.

[0019] In one optional implementation, a distributed inference cluster is established on the master device and the slave device, including: controlling the master device to start the head node service of the distributed inference cluster, and determining the cluster address of the distributed inference cluster by the local address of the master device; based on the node information of the distributed inference cluster, constructing a communication channel between the master device and the slave device, and starting the inference service of the distributed inference cluster; controlling the slave device to start the worker node service of the distributed inference cluster, and joining the distributed inference cluster corresponding to the master device; controlling the slave device to connect to the master device based on the communication channel, and waiting for the master device to schedule.

[0020] The method for building a distributed inference cluster provided in this invention involves the master device automatically starting the head node service and defining the cluster address of the distributed inference cluster, while the slave devices automatically start worker nodes to connect. The entire process requires no manual writing or modification of cluster configuration files. The master device automatically establishes a communication channel with the slave devices, enabling the master device to uniformly schedule the slave devices and quickly launch the distributed inference service. This allows multiple terminal devices to directly connect and build a distributed inference cluster, meeting the real-time inference service requirements at the edge.

[0021] Secondly, the present invention provides a device for constructing a distributed inference cluster, comprising: an acquisition module for acquiring a first external network connectivity status of a first terminal device; a message listening module for acquiring a link layer broadcast message of a second terminal device based on a message listening thread of the first terminal device, wherein the second terminal device is the peer device of the first terminal device; the first terminal device and the second terminal device exchanging information based on the link layer broadcast message; a message parsing module for parsing the link layer broadcast message to acquire device information of the second terminal device, wherein the device information includes a second external network connectivity status; a master-slave device determination module for determining a master device and a slave device from the first terminal device and the second terminal device based on the first external network connectivity status and the second external network connectivity status; and a cluster construction module for constructing a distributed inference cluster on the master device and the slave device in response to the master device and the slave device passing connectivity verification.

[0022] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method for constructing a distributed inference cluster as described in the first aspect or any corresponding embodiment.

[0023] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the method for constructing a distributed inference cluster according to the first aspect or any corresponding embodiment described above.

[0024] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the method for constructing a distributed inference cluster according to the first aspect or any corresponding embodiment described above. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first method for constructing a distributed inference cluster according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a second method for constructing a distributed inference cluster according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the third process for constructing a distributed inference cluster according to an embodiment of the present invention; Figure 5 This is a structural block diagram of a device for building a distributed inference cluster according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0029] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0030] Before providing a further detailed description of the embodiments of the present invention, some of the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0031] MAC: Media Access Control, a unique hardware identifier for a network interface card (NIC).

[0032] UDP: User Datagram Protocol, a connectionless transport layer protocol.

[0033] DHCP: Dynamic Host Configuration Protocol, used to automatically assign IP addresses.

[0034] vLLM: Open source distributed inference engine for large language models Ray: An open-source distributed computing framework for distributed AI task scheduling.

[0035] NCCL: NVIDIA Collective Communications Library, used for data synchronization between multiple GPUs / nodes.

[0036] Master device: The node device responsible for global control in a distributed inference cluster.

[0037] Slave device: The worker node device in a distributed inference cluster responsible for executing computational tasks.

[0038] In edge computing and edge deployment scenarios, such as small AI workstations, emergency computing power in the field, and offline demonstration scenarios, there is usually no pre-set network environment: there is neither a DHCP server to automatically assign IP addresses nor professional maintenance personnel to configure them on-site. For this scenario, current distributed inference cluster deployment solutions rely on manual pre-configuration of IP addresses, assignment of master and slave roles, and modification of configuration files. The operation threshold is high and the process is cumbersome, failing to achieve the plug-and-play experience of "simply connect the network cable to form a network".

[0039] Current self-organizing network technologies in the field are mainly aimed at low-bandwidth IoT scenarios, such as Zigbee and Bluetooth Mesh, which cannot meet the high bandwidth and low latency requirements of large model inference. On the other hand, TCP / IP-based self-organizing network solutions either rely on DHCP for centralized address allocation or require manual pre-configuration of node information, which cannot achieve true zero-configuration automatic networking.

[0040] The automatic networking based on a DHCP server relies on a DHCP server deployed in the network to automatically allocate IP addresses. The specific process is as follows: all devices connect to the same local area network (LAN), and the DHCP server pre-configures an IP address pool; after powering on, the devices send a DHCP Discover broadcast message requesting an IP address, and the DHCP server replies with the allocated IP address; maintenance personnel determine the roles of master and slave nodes through preset configuration or manual specification; maintenance personnel manually modify the distributed inference engine's configuration file, fill in the IP addresses of each node, and start the cluster service. This networking method relies on a centralized DHCP server, which must be pre-deployed in the network, making it unusable in scenarios where two devices are directly connected without a central hub; DHCP can only allocate IP addresses and cannot automatically complete master / slave role election, still requiring manual intervention in role allocation; moreover, it is based solely on hardware identifier sorting and cannot optimize master node selection based on business needs (such as external network access capabilities); after IP address allocation, manual configuration of the distributed inference service is still required, failing to achieve full automation from physical connection to inference service readiness.

[0041] The manual static configuration networking method involves operations and maintenance personnel manually configuring static IP addresses and assigning master-slave roles to each device. The specific process is as follows: operations and maintenance personnel pre-plan cluster IP addresses, log in to each device to configure static IPs; manually designate one device as the master node and the rest as slave nodes; modify the distributed inference configuration files of each node, filling in the peer IP address; manually start the services of each node, verify cluster connectivity, and complete the deployment. However, this method requires users to have professional knowledge of network configuration and distributed system operations and maintenance, which is beyond the capabilities of ordinary users, resulting in a very high deployment threshold. The manual configuration process is cumbersome, repetitive, and time-consuming, prone to problems such as IP address conflicts and incorrect role configurations, leading to low efficiency. After devices are powered on again or connections are changed, the entire configuration process needs to be re-executed, making automatic networking at startup and plug-and-play difficult. When adding or removing nodes, the information of all nodes needs to be reconfigured, resulting in high maintenance costs and poor scalability.

[0042] Based on this, the technical solution of this invention achieves information exchange based on link layer broadcast, does not rely on proprietary protocols, and is compatible with all standard Ethernet devices; it can automatically complete cluster formation with only external network connectivity, significantly reducing the deployment threshold; it uses external network connectivity to determine master and slave devices, ensuring that the master device has external network access capability; after the master and slave devices pass connectivity verification, it automatically forms a distributed inference cluster, realizing full-process automation of the distributed inference cluster, without manual operation, and can adapt to rapid networking in edge inference scenarios such as no maintenance and no preset network, significantly reducing the deployment operation threshold of distributed inference clusters among multiple terminal devices, improving the deployment efficiency of distributed inference clusters, and achieving plug-and-play networking.

[0043] As an optional application scenario of this invention, such as Figure 1 As shown, this application scenario may include at least one electronic device and at least one server. Figure 1 The example illustrates that the application scenario includes a computer 101, a mobile terminal 102, and a server 103, and that electronic devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 via a network 110.

[0044] The electronic device integrates an AI intelligent agent box, which can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0045] According to an embodiment of the present invention, a method for constructing a distributed inference cluster is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] This embodiment provides a method for constructing a distributed inference cluster. Figure 2 This is a flowchart of a method for constructing a distributed inference cluster according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the first external network connectivity status of the first terminal device.

[0047] The first external network connectivity status is used to characterize whether the first terminal device has the ability to access the Internet. The first external network connectivity status can include external network connected and external network not connected.

[0048] Specifically, the first terminal device acts as an independent edge device, on which an intelligent agent box is deployed. This intelligent agent box integrates AI computing hardware, pre-trained large models, and customized intelligent agent applications. The first terminal device is equipped with an Ethernet interconnection interface, and by continuously monitoring the physical link status of the designated Ethernet interconnection interface, the connection status of the first external network corresponding to the first terminal device is obtained.

[0049] Step S202: Based on the message listening thread of the first terminal device, obtain the link layer broadcast message of the second terminal device.

[0050] The second terminal device is the peer device of the first terminal device, and the first terminal device and the second terminal device exchange information based on link layer broadcast messages.

[0051] Link-layer broadcast messages are messages sent to the link-layer broadcast address and are used to perform normal broadcast message transmission when the device is not configured with an IP address.

[0052] When the first terminal device and the second terminal device exchange information via link layer broadcast messages, the first terminal device will send the link layer broadcast messages to a designated port so that the first terminal device can receive the link layer broadcast messages sent by the second terminal device from the designated port, and the second terminal device can receive the link layer broadcast messages sent by the first terminal device from the designated port.

[0053] The first terminal device has an internal network interface. By starting the packet listening thread corresponding to the internal network interface, it can obtain the link layer broadcast packets sent by the second terminal device to the specified port.

[0054] In a specific example, without configuring an IP address, the first terminal device can exchange information with the second terminal device via UDP broadcast at the link layer. Specifically, the first terminal device constructs a UDP broadcast message, which includes three key fields: the MAC address of its local Ethernet interface, the local hostname (whose format can be cube-XXXX, where XXXX is the last four hexadecimal characters of the LocalMAC address used for validity verification), and the local external network connectivity status NetStatus.

[0055] Setting the socket option SO_BROADCAST for broadcast messages enables message broadcasting, sending UDP broadcast messages to the link-layer broadcast address 255.255.255.255. Therefore, even if the first terminal device is not configured with an IP address, its Linux kernel will allow the normal transmission of such broadcast messages.

[0056] Step S203: Parse the link layer broadcast message to obtain the device information of the second terminal device, including the second external network connectivity status.

[0057] After receiving the link layer broadcast message sent by the second terminal device through the message listening thread, the first terminal device parses the link layer broadcast message based on the message encapsulation protocol of the link layer broadcast message and extracts the second external network connectivity status corresponding to the second terminal device.

[0058] Step S204: Based on the first external network connectivity status and the second external network connectivity status, determine the master device and the slave device from the first terminal device and the second terminal device.

[0059] Combining the first and second external network connectivity statuses, a master-slave device role election is triggered. Specifically, the external network access capability represented by the external network connectivity status is used as the priority rule for master-slave devices. For example, if the first terminal device has external network access capability and the second terminal device does not, then the first terminal device is determined to be the master device and the second terminal device is determined to be the slave device.

[0060] Step S205: In response to the master device and slave device passing connectivity verification, a distributed inference cluster is established on the master device and slave device.

[0061] The system automatically configures network addresses for both the master and slave devices and verifies connectivity based on these configurations. Once connectivity is confirmed between the master and slave devices, the head node and worker node services of the Ray framework are automatically started to control the formation of the distributed inference cluster. Subsequently, the master device starts the inference service of the distributed inference cluster, and the slave devices can wait for scheduling by the master device to enter the inference working state.

[0062] The distributed inference cluster construction method provided in this embodiment involves information exchange between the first and second terminal devices via link-layer broadcast messages. Since there is no central information interaction, the external network connectivity status and device identifiers of both parties can be automatically collected without the need for a DHCP server. Based on the external network connectivity status, the master-slave role election is automatically completed between the first and second terminal devices, eliminating the need for manual configuration. After the master-slave connectivity verification is successful, a distributed inference cluster is automatically built on the master and slave devices. This achieves zero-configuration automatic networking and inference between multiple terminal devices, enabling rapid networking in edge inference scenarios such as networks without maintenance. It significantly reduces the deployment threshold of distributed inference clusters between multiple terminal devices, improves deployment efficiency, and facilitates plug-and-play networking.

[0063] This embodiment provides a method for constructing a distributed inference cluster. Figure 3 This is a flowchart of a method for constructing a distributed inference cluster according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the first external network connectivity status of the first terminal device.

[0064] Specifically, step S301 includes: Step S3011: Detect the network link status corresponding to the network interface of the first terminal device.

[0065] The network interface is the interface through which the first terminal device connects to the external network, such as the Ethernet cable interface or wireless port; the network link status is used to characterize whether the physical link between the first terminal device and the external network is connected.

[0066] Specifically, after startup, the first terminal device continuously monitors the physical link status of the network interface to determine whether the network cable is connected or whether wireless access has been completed. If the network cable is connected or wireless access to the external network is successful, the network link status can be determined to be connected (i.e., up); otherwise, the network link status is determined to be disconnected (i.e., down).

[0067] In step S3012, in response to the network link status being connected, the message discovery window of the first terminal device is activated, and the first external network connectivity status of the first terminal device is detected.

[0068] When the network link status changes from disconnected to connected, the first terminal device initiates a message discovery window of a preset duration to receive discovery messages from the peer device (i.e., the second terminal device). Simultaneously, it checks its external network connectivity by pinging a public IP address (e.g., 8.8.8.8). If the first terminal device can ping the public IP address, it can be determined that it currently has internet access capability; if it cannot ping the public IP address, it can be determined that it currently does not have internet access capability. After obtaining the first external network connectivity status, the connectivity result represented by this status is stored locally in the NetStatus field. A NetStatus field of True indicates the presence of an external network, i.e., the external network is connected; a NetStatus field of False indicates the absence of an external network, i.e., the external network is not connected.

[0069] Step S302: Based on the message listening thread of the first terminal device, obtain the link layer broadcast message of the second terminal device. The second terminal device is the peer device of the first terminal device, and the first terminal device and the second terminal device exchange information based on the link layer broadcast message. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0070] Step S303: Parse the link layer broadcast message to obtain the device information of the second terminal device, including the second external network connectivity status.

[0071] Specifically, step S303 includes: In step S3031, in response to the link layer broadcast message being a User Datagram Protocol (UDP) message, the UDP message is parsed based on the UDP to obtain the second external network connectivity status of the second terminal device.

[0072] If the link layer broadcast message is a User Datagram Protocol (UDP) message encapsulated based on the UDP protocol, after the first terminal device listens to the UDP message sent by the second terminal device to the specified port, it can parse the UDP message based on the UDP protocol to obtain the second external network connectivity status and the physical address of the second terminal device (i.e., the MAC address of the second terminal device), and use the physical address of the second terminal device as the identifier of the second device.

[0073] In step S3032, in response to the link layer broadcast message being a Dynamic Host Configuration Protocol (DHCP) discovery message, the additional information field of the DHCP discovery message is parsed based on DHCP to obtain the second external network connectivity status corresponding to the additional information field.

[0074] If the link layer broadcast message is a Dynamic Host Configuration Protocol Discovery (DHCP Discover) message encapsulated based on the DHCP protocol, after the first terminal device obtains the DHCP Discover message sent by the second terminal device to the specified port, it can parse the DHCP Discover message based on the DHCP protocol to extract the additional information corresponding to the Option field and the physical address of the second terminal device (i.e., the MAC address of the second terminal device).

[0075] Extract the hostname field of the second terminal device and the external network connectivity status field corresponding to the second terminal device from the additional information field to obtain the second external network connectivity status corresponding to the second terminal device.

[0076] Here, by utilizing DHCP's native packet sending behavior, there is no need for the first terminal device to construct additional UDP packets, resulting in stronger adaptability.

[0077] In step S3033, in response to the link layer broadcast message being a link layer discovery protocol message, the type-length-value field of the link layer discovery protocol message is parsed based on the link layer discovery protocol to obtain the second external network connectivity status of the second terminal device.

[0078] If the link layer broadcast message is a Link Layer Discovery Protocol (LLDP) message encapsulated based on the Link Layer Discovery Protocol (LLDP), after the first terminal device listens to the LLDP message sent by the second terminal device to the specified port, it can parse the LLDP message based on the LLDP protocol and extract the type-length-value field (TLV field) from the message.

[0079] By parsing the TLV field, the second external network connectivity status and the physical address (i.e., the MAC address) of the second terminal device can be obtained.

[0080] Using standard link layer discovery protocols for message broadcasting makes the discovery of second terminal devices more standardized and more compatible.

[0081] In some optional implementations, the above method further includes: Step a1: Parse the link layer broadcast message to obtain the device name of the second terminal device. The device information also includes the device name.

[0082] Step a2: In response to the device name passing the validity check, the device information of the second terminal device is stored in the neighbor node mapping table corresponding to the first terminal device.

[0083] In step a3, in response to the device name failing the validity check, the link layer broadcast message of the second terminal device is discarded.

[0084] The first terminal device parses the link-layer broadcast message to obtain the hostname of the second terminal device carried in the link-layer broadcast message. Regular expression matching is then performed on the hostname of the second terminal device to verify whether the hostname conforms to the cube-XXXX convention.

[0085] If the hostname of the second terminal device conforms to the cube-XXXX format, then the hostname of the second terminal device can be determined to have passed the validity check. At this time, the device information of the second terminal device is stored in the neighbor node mapping table corresponding to the first terminal device. For example, the structure of the neighbor node mapping table can be: {Device ID (e.g., MAC): (Device Name Hostname, External Network Connectivity Status NetStatus)}.

[0086] If the hostname of the second terminal device does not conform to the cube-XXXX convention, it can be determined that the hostname of the second terminal device has failed the validity check. In this case, the first terminal device can discard the link layer broadcast messages of the second terminal device that it has been listening to.

[0087] In the above implementation, by parsing device names from link-layer broadcast messages and using these names for validity verification, illegal and abnormal link-layer broadcast messages are filtered out, preventing malicious messages from interfering with the networking process of the distributed inference cluster and improving network stability. Furthermore, by storing the device information that has passed the validity verification in a neighbor node mapping table, it is easy to directly look up and reuse the table later without repeated packet capture and parsing, further improving network response speed.

[0088] Step S304: Based on the first external network connectivity status and the second external network connectivity status, determine the master device and the slave device from the first terminal device and the second terminal device. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above, which will not be repeated here.

[0089] Step S305: In response to the master device and slave device passing connectivity verification, a distributed inference cluster is established on the master device and slave device. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above, which will not be repeated here.

[0090] The distributed inference cluster construction method provided in this embodiment uses the network link connectivity status as the trigger point to precisely control the timing of the first terminal device's initiation of external network connectivity. Simultaneously, the packet discovery window is only opened when the network link is in a connected state, avoiding invalid packet capture detection when the network link is disconnected, thus saving computing resources on the first terminal device. When the network link is in a connected state, external network status detection is simultaneously initiated to obtain the basic network capabilities of the first terminal device, providing a preliminary judgment basis for subsequent master-slave election and reducing invalid computation.

[0091] By being compatible with three types of link-layer broadcast messages—User Datagram Protocol (UDP), Dynamic Host Configuration Protocol (DHCP), and Link Layer Discovery Protocol (LLDM)—and without relying on proprietary protocols, it can adapt to different first-end device reporting methods, thus improving versatility. Furthermore, relying solely on Layer 2 link-layer broadcast messages for information exchange eliminates the need for pre-allocation of Layer 3 network addresses, enabling rapid acquisition of device information from peer devices even without a central network.

[0092] This embodiment provides a method for constructing a distributed inference cluster. Figure 4 This is a flowchart of a method for constructing a distributed inference cluster according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the first external network connectivity status of the first terminal device. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0093] Step S402: Based on the message listening thread of the first terminal device, obtain the link layer broadcast message of the second terminal device. The second terminal device is the peer device of the first terminal device, and the first terminal device and the second terminal device exchange information based on the link layer broadcast message. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments, which will not be repeated here.

[0094] Step S403: Parse the link layer broadcast message to obtain the device information of the second terminal device, including the second external network connectivity status. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments; they will not be repeated here.

[0095] Step S404: Based on the first external network connectivity status and the second external network connectivity status, determine the master device and the slave device from the first terminal device and the second terminal device.

[0096] Specifically, step S404 includes: Step S4041: Determine whether the first external network connectivity status and the second external network connectivity status are consistent.

[0097] The first external network connectivity status corresponding to the first terminal device is compared with the second external network connectivity status corresponding to the second terminal device to determine whether the external network connectivity statuses of the first and second terminal devices are consistent. If the first and second external network connectivity statuses are consistent, it means that both the first and second terminal devices have external network access capabilities, and in this case, a master-slave role election is required based on the device identifier; if the first and second external network connectivity statuses are inconsistent, it means that one of the first and second terminal devices does not have external network access capabilities, and in this case, a master-slave role election is performed based on the external network connectivity status.

[0098] Step S4042: If the first external network connectivity status and the second external network connectivity status are the same, then obtain the first device identifier corresponding to the first terminal device and the second device identifier corresponding to the second terminal device respectively.

[0099] If the first external network connectivity status and the second external network connectivity status are the same, it means that both the first terminal device and the second terminal device have external network access capabilities, or neither the first terminal device nor the second terminal device has external network access capabilities. That is, the NetStatus field for the external network connectivity status of the first terminal device and the second terminal device is either True or False.

[0100] The first device identifier is an identifier that uniquely identifies the first terminal device, and the second device identifier is an identifier that uniquely identifies the second terminal device. The first device identifier and the second device identifier can be MAC address, serial number (SN), etc.

[0101] The first terminal device has corresponding configuration information, in which a first device identifier is set. The first terminal device obtains the corresponding first device identifier by reading the configuration information.

[0102] The first terminal device and the second terminal device exchange information based on link layer broadcast messages. The first terminal device can parse the link layer broadcast message sent by the second terminal device to extract the second device identifier corresponding to the second terminal device.

[0103] Step S4043: Convert the first device identifier and the second device identifier into the first and second values ​​of preset bytes.

[0104] At this point, using the device identifier as a fallback rule for master-slave device decision-making, the first device identifier is converted into a first value of a preset byte, and the second device identifier is converted into a second value of a preset byte. Here, the preset byte is a pre-defined set of value conversion bytes, such as 6 bytes; the first and second values ​​can be hexadecimal integers.

[0105] Taking MAC address as an example, MAC address is a fixed 48-bit (6-byte) format: 6 hexadecimal segments, each segment is 1 byte, for a total of 6 bytes; it is then shifted and concatenated from left to right (highest byte first) to form a complete integer, directly converting the original binary into an unsigned decimal integer.

[0106] For example, the MAC address of the first terminal device is 00:11:22:33:44:55; the MAC address of the second terminal device is 00:11:22:33:44:66. Breaking each MAC address down into single bytes, we get the single bytes for the first terminal device as 0x00, 0x11, 0x22, 0x33, 0x44, 0x55; and the single bytes for the second terminal device as 0x00, 0x11, 0x22, 0x33, 0x44, 0x66.

[0107] Using decimal numerical formulas: val = B0×2 40 + B1×2 32 + B2×2 24 + B3×2 16 + B4×2 8 + B5.

[0108] Among them, B0, B1, B2, B3, B4 and B5 are each single byte.

[0109] For each single byte, decimal values ​​are obtained, and the decimal numerical formulas mentioned above are used for item-by-item calculations to obtain the corresponding unsigned integers.

[0110] Convert each byte of the first terminal device into a decimal value and calculate the first value using the above formula; convert each byte of the second terminal device into a decimal value and calculate the second value using the above formula.

[0111] Step S4044: Based on the comparison result of the first value and the second value, determine the master device and the slave device from the first terminal device and the second terminal device.

[0112] The first value is compared with the second value to determine the comparison result. Then, the master / slave device is elected according to this comparison result.

[0113] For example, if the comparison result indicates that the first value is greater than the second value, then the first terminal device corresponding to the first value can be identified as the master device, and the second terminal device corresponding to the second value can be identified as the slave device.

[0114] For example, if the comparison result indicates that the first value is less than the second value, then the first terminal device corresponding to the first value can be identified as the master device, and the second terminal device corresponding to the second value can be identified as the slave device.

[0115] Therefore, it can ensure that nodes with external network access capabilities become the master devices first (to facilitate the master devices to complete external network-related businesses such as model updates and cloud collaboration), and can obtain a deterministic election result based on a unique device identifier when the external network status of both parties is consistent, thus completely avoiding the split-brain problem.

[0116] Step S4045: If the first external network connectivity status indicates that the first terminal device has external network access capability, and the second external network connectivity status indicates that the second terminal device does not have external network access capability, then the first terminal device is determined as the master device, and the second terminal device is determined as the slave device.

[0117] If the first external network connectivity status indicates that the first terminal device has external network access capability, while the second external network connectivity status indicates that the second terminal device does not have external network access capability, that is, the NetStatus field of the external network connectivity status corresponding to the first terminal device is True and the NetStatus field of the external network connectivity status corresponding to the second terminal device is False, then the first terminal device is determined to be the master device and the second terminal device is determined to be the slave device.

[0118] Step S4046: If the first external network connectivity status indicates that the first terminal device does not have external network access capability, while the second external network connectivity status indicates that the second terminal device has external network access capability, then the second terminal device is determined as the master device, and the first terminal device is determined as the slave device.

[0119] If the first external network connectivity status indicates that the first terminal device does not have external network access capability, while the second external network connectivity status indicates that the second terminal device has external network access capability, that is, the NetStatus field of the external network connectivity status corresponding to the first terminal device is False, and the NetStatus field of the external network connectivity status corresponding to the second terminal device is True, then the second terminal device is determined as the master device, and the first terminal device is determined as the slave device.

[0120] The distributed inference cluster construction method provided in this embodiment, when the external network connectivity status of the first terminal device and the second terminal device is consistent, converts the device identifiers of the first terminal device and the second terminal device into preset byte values ​​for comparison. The determination rules are unified, the calculation logic is simple, and it is easy to implement at the device level. Hierarchical election is performed using external network connectivity status as the first priority and device identifier value as the second priority, ensuring that the determination of master and slave devices has a unified and quantifiable standard, and avoiding master-slave device election conflicts.

[0121] External network access capability, as a representation of external network connectivity, is used as the primary criterion for determining the master device. Devices with external network access are prioritized as the head nodes of the distributed inference cluster, facilitating the execution of external network-related services such as model updates and remote scheduling. By differentiating the allocation of master and slave devices with and without external network access, the actual business needs of edge clusters are aligned, improving the inference service adaptability of the distributed inference cluster.

[0122] Step S405: In response to the master device and slave device passing connectivity verification, a distributed inference cluster is established on the master device and slave device.

[0123] Specifically, step S405 includes: Step S4051: Control the master device to start the head node service of the distributed inference cluster, and determine the cluster address of the distributed inference cluster by the local address of the master device.

[0124] After confirming that the master and slave devices have passed connectivity verification, the master device starts the Ray framework's head node service, listening on control port Y (e.g., port 6379). Simultaneously, it starts the vLLM inference service, sets the tensor parallelism to the number of nodes in the distributed inference cluster, and specifies the cluster address of the distributed inference cluster as the master device's local address, which is the corresponding network address of the master device.

[0125] Step S4052: Based on the node information of the distributed inference cluster, establish a communication channel between the master device and the slave device, and start the inference service of the distributed inference cluster.

[0126] The master device uses vLLM to synchronize node information of the distributed inference cluster and establishes an NCCL communication channel between the master and slave devices. Then, it loads the large model weight file, starts the inference service of the distributed inference cluster, and completes the setup of the distributed inference cluster.

[0127] Step S4053: Control the slave device to start the worker node service of the distributed inference cluster, and add the slave device to the distributed inference cluster corresponding to the master device.

[0128] After confirming that the master device and slave device have passed the connectivity verification, the slave device starts the Ray framework worker node service, connects to the control port Y (such as port 6379) of the master device's network address, and automatically joins the distributed inference cluster built by the master device.

[0129] Step S4054: Control the slave device to connect to the master device based on the communication channel, and wait for the master device to schedule.

[0130] After a slave device joins the distributed inference cluster, it loads the model shards corresponding to the distributed inference cluster and connects to the NCCL communication channel to wait for the master device to execute inference service scheduling and control the slave device to enter the inference working state.

[0131] The method for building a distributed inference cluster provided in this invention involves the master device automatically starting the head node service and defining the cluster address of the distributed inference cluster, while the slave devices automatically start worker nodes to connect. The entire process requires no manual writing or modification of cluster configuration files. The master device automatically establishes a communication channel with the slave devices, enabling the master device to uniformly schedule the slave devices and quickly launch the distributed inference service. This allows multiple terminal devices to directly connect and build a distributed inference cluster, meeting the real-time inference service requirements at the edge.

[0132] In some alternative implementations, connectivity verification is performed on the master and slave devices, including: Step b1: Check whether the master device is configured with the first master address in the target network segment, and whether the slave device is configured with the first slave address in the target network segment.

[0133] Step b2: If the master device is configured with a first master address and the slave device is configured with a first slave address, then the connectivity of the master device and the slave device is verified based on the first master address and the first slave address.

[0134] After the master-slave device role election is completed, connectivity verification is performed on both devices. Specifically, it first checks whether the master device is configured with the first master address within the target network segment, and whether the slave device is configured with the first slave address within the target network segment. The target network segment is 192.168.124.0 / 24, the first master address is a valid network address (IP address) pre-configured for the master device, and the first slave address is a valid network address (IP address) pre-configured for the slave device.

[0135] If it is detected that the master device is pre-configured with the first master address in the target network segment and the slave device is pre-configured with the first slave address in the target network segment, then the network address configuration of the master and slave devices will be directly targeted, and the bidirectional connectivity verification between the master and slave devices will be directly executed.

[0136] Step b3: If the master device is not configured with a first master address and / or the slave device is not configured with a first slave address, then configure a preset second master address for the master device and a preset second slave address for the slave device.

[0137] Step b4: Based on the second master address and the second slave address, perform connectivity verification on the master device and the slave device.

[0138] If it is detected that the master device is not configured with the first master address in the target network segment, or the slave device is not configured with the first slave address in the target network segment, or neither the master device nor the slave device is configured with the first master address, then a static network address is automatically configured based on the elected master device. For example, the second master address of the master device is configured as 192.168.124.1 with a subnet mask of 255.255.255.0; and the second slave address of the slave device is configured as 192.168.124.2 with a subnet mask of 255.255.255.0.

[0139] By combining the automatically configured second master address and second slave address, bidirectional connectivity verification between the master and slave devices is performed.

[0140] Specifically, the bidirectional connectivity verification steps mentioned above may include: the master device pinging the network address corresponding to the slave device, and the slave device pinging the network address corresponding to the master device; if the master device can ping the slave device and the slave device can ping the master device, the connectivity between the master and slave devices is determined to be normal, and the subsequent distributed inference cluster deployment is initiated; if the master device cannot ping the slave device, or the slave device cannot ping the master device, the process is retried 3 times, with a 1-second interval between each retry; after a failed retry, the network address configuration of the master and slave devices is reset, and the connectivity verification process is retried.

[0141] The distributed inference cluster construction method provided in this embodiment, when detecting that the first terminal device and the second terminal device are configured with network addresses of the target network segment, performs connectivity verification based on the configured network addresses, eliminating the need for repeated address allocation, reusing existing network configurations on-site, and shortening network construction time. Bidirectional connectivity verification based on network addresses of the same network segment accurately identifies network faults such as port, link, and network segment disconnections, ensuring the reliability of communication links for the subsequently constructed distributed inference cluster.

[0142] When the first terminal device and / or the second terminal device do not have a network address in the target network segment, a network address is automatically assigned, achieving self-assignment of addresses without a DHCP server and truly realizing zero-configuration of network addresses. At the same time, by automatically assigning a network address in the target network segment, the problem of devices being unable to interconnect due to network address mismatch can be avoided, improving the compatibility of ad hoc networks.

[0143] In some optional implementations, the above method further includes: Step c1: In response to the existence of multiple slave devices, obtain the third device identifier corresponding to each slave device.

[0144] Step c2: Convert each third device identifier into a third value of a preset byte.

[0145] Step c3: According to the sorting result of the third value, configure the preset second slave address for each slave device in turn.

[0146] When there are multiple slave devices, the device identifier (i.e., the third device identifier) ​​corresponding to each slave device is obtained in sequence, and each third device identifier is converted into a third value of preset bytes. The conversion method is the same as the conversion process of the first value and the second value, which will not be described here.

[0147] Sort the various third values ​​to obtain a sorted result. According to the sorted result, configure the corresponding second slave address 192.168.124.1~192.168.124.N for each slave device in an orderly manner.

[0148] The distributed inference cluster construction method provided in this embodiment converts the third device identifier corresponding to each slave device into a third value of preset bytes for multiple slave devices. The network addresses of each slave device are then allocated in an orderly manner according to the third value to avoid network address allocation conflicts. Furthermore, the allocation of network addresses to each slave device according to the sorting result of the third value ensures that the allocation results of slave device network addresses are consistent each time the network is formed, facilitating device management, device location, and maintenance troubleshooting within the distributed inference cluster.

[0149] In some optional implementations, the above method further includes: Step d1: In response to the failure of both the master and slave devices to pass connectivity verification, obtain the number of connectivity verification failures.

[0150] In step d2, if the number of connectivity verification failures does not exceed the preset value, then the connectivity verification of the master device and the slave device is restarted at the preset interval.

[0151] In step d3, in response to the number of connectivity verification failures exceeding a preset value, the network addresses of the master and slave devices are reset, and connectivity verification is performed based on the reset network addresses.

[0152] When the master and slave devices fail connectivity verification, the number of connectivity verification failures is counted. This number of failures is compared with a preset value (e.g., 3 times) to determine if it exceeds the preset value. If the number of failures does not exceed the preset value, the connectivity verification between the master and slave devices is restarted at preset intervals (e.g., 0.5 seconds, 1 second, etc.).

[0153] If the number of connectivity verification failures exceeds the preset value, the network addresses of the target network segment will be reassigned to the master and slave devices, and connectivity verification between the master and slave devices will be performed according to the reset network addresses.

[0154] The distributed inference cluster construction method provided in this invention employs an interval retry mechanism after connectivity verification failures. This allows for automatic recovery from occasional fault scenarios such as temporary network jitter and packet loss, without requiring manual restart of the network setup process. After the number of failures reaches a preset value, the network address is automatically reset and connectivity verification is re-executed, avoiding persistent network faults such as network segment conflicts and address occupancy. This enhances self-healing capabilities and reduces on-site manual intervention.

[0155] This embodiment also provides a device for building a distributed inference cluster, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0156] This embodiment provides a device for building a distributed inference cluster, such as... Figure 5 As shown, it includes: The acquisition module 501 is used to acquire the first external network connectivity status of the first terminal device.

[0157] The message listening module 502 is used to obtain the link layer broadcast message of the second terminal device based on the message listening thread of the first terminal device. The second terminal device is the peer device of the first terminal device. The first terminal device and the second terminal device exchange information based on the link layer broadcast message.

[0158] The message parsing module 503 is used to parse link layer broadcast messages and obtain device information of the second terminal device, including the second external network connectivity status.

[0159] The master-slave device determination module 504 is used to determine the master device and the slave device from the first terminal device and the second terminal device based on the first external network connectivity status and the second external network connectivity status.

[0160] Cluster building module 505 is used to build a distributed inference cluster on the master device and slave device in response to the master device and slave device passing connectivity verification.

[0161] In some alternative implementations, the message parsing module 503 includes: The first parsing unit is used to parse the User Datagram Protocol (UDP) message in response to a link layer broadcast message being a UDP message, and to obtain the second external network connectivity status of the second terminal device based on the UDP message.

[0162] The second parsing unit is used to respond to the link layer broadcast message as a Dynamic Host Configuration Protocol (DHCP) discovery message, and then, based on DHCP, parse the additional information field of the DHCP discovery message to obtain the second external network connectivity status corresponding to the additional information field.

[0163] The third parsing unit is used to respond to the link layer broadcast message as a link layer discovery protocol message, and then parse the type-length-value field of the link layer discovery protocol message to obtain the second external network connectivity status of the second terminal device.

[0164] In some optional implementations, the message parsing module 503 further includes: The device name acquisition unit is used to parse the link layer broadcast message and obtain the device name of the second terminal device. The device information also includes the device name.

[0165] The information storage unit is used to store the device information of the second terminal device in the neighbor node mapping table corresponding to the first terminal device in response to the device name passing the validity verification.

[0166] The message discarding unit is used to discard the link layer broadcast message of the second terminal device in response to the device name failing the validity check.

[0167] In some optional implementations, the master-slave device determination module 504 includes: The judgment unit is used to determine whether the first external network connectivity status and the second external network connectivity status are consistent.

[0168] The device identifier acquisition unit is used to acquire the first device identifier corresponding to the first terminal device and the second device identifier corresponding to the second terminal device if the first external network connectivity status and the second external network connectivity status are the same.

[0169] The identifier conversion unit is used to convert the first device identifier and the second device identifier into a first value and a second value of preset bytes.

[0170] The first master-slave election unit is used to determine the master device and the slave device from the first terminal device and the second terminal device based on the comparison result of the first value and the second value.

[0171] The second master-slave election unit is used to determine the first terminal device as the master device and the second terminal device as the slave device if the first external network connectivity status indicates that the first terminal device has external network access capability, and the second external network connectivity status indicates that the second terminal device does not have external network access capability.

[0172] The third master-slave election unit is used to determine the second terminal device as the master device and the first terminal device as the slave device if the first external network connectivity status indicates that the first terminal device does not have external network access capability, and the second external network connectivity status indicates that the second terminal device has external network access capability.

[0173] In some alternative embodiments, the above-described apparatus further includes: The verification module is used to verify the connectivity of the master and slave devices.

[0174] Specifically, the verification module includes: The address configuration detection unit is used to detect whether the master device is configured with the first master address in the target network segment, and whether the slave device is configured with the first slave address in the target network segment.

[0175] The first connectivity verification unit is used to verify the connectivity of the master device and the slave device based on the first master address and the first slave address if the master device is configured with a first master address and the slave device is configured with a first slave address. The address allocation unit is used to configure a preset second master address for the master device and a preset second slave address for the slave device if the master device is not configured with a first master address and / or the slave device is not configured with a first slave address.

[0176] The second connectivity verification unit is used to verify the connectivity of the master device and the slave device based on the second master address and the second slave address.

[0177] In some alternative implementations, the address allocation unit includes: The device identifier acquisition subunit is used to acquire the third device identifier corresponding to each slave device if there are multiple slave devices.

[0178] The conversion subunit is used to convert each third device identifier into a third value of a preset byte.

[0179] The allocation subunit is used to configure a preset second slave address for each slave device in turn according to the sorting result of the third value.

[0180] In some optional implementations, the verification module further includes: The count acquisition unit is used to acquire the number of connectivity verification failures in response to the master device and slave device failing the connectivity verification.

[0181] The restart unit is used to restart the connectivity verification of the master device and the slave device at a preset interval if the number of connectivity verification failures does not exceed a preset value.

[0182] The address reset unit is used to reset the network addresses of the master device and the slave device in response to the number of connectivity verification failures exceeding a preset value, and to perform connectivity verification based on the reset network addresses.

[0183] In some alternative implementations, the cluster assembly module 505 includes: The cluster address determination unit is used to control the master device to start the head node service of the distributed inference cluster and determine the cluster address of the distributed inference cluster from the local address of the master device.

[0184] The inference service deployment unit is used to build a communication channel between the master device and the slave device based on the node information of the distributed inference cluster, and to start the inference service of the distributed inference cluster.

[0185] The slave device join unit is used to control the slave device to start the worker node service of the distributed inference cluster and join the slave device to the distributed inference cluster corresponding to the master device.

[0186] The scheduling and waiting unit is used to control the slave device to connect to the master device based on the communication channel and wait for the master device to schedule it.

[0187] The distributed inference cluster building apparatus provided in this embodiment of the invention can execute the distributed inference cluster building method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0188] The first and second terminal devices exchange information via link-layer broadcast messages, without a central information exchange. Therefore, the external network connectivity status of both parties can be automatically collected without setting up a DHCP server. Based on the external network connectivity status, the master and slave roles are automatically elected between the first and second terminal devices, without the need for manual configuration. After the connectivity verification of the master and slave devices passes, a distributed inference cluster is automatically built on the master and slave devices, realizing zero-configuration automatic networking and inference between multiple terminal devices. This enables rapid networking in edge inference scenarios such as no maintenance, significantly reducing the deployment and operation threshold of distributed inference clusters between multiple terminal devices, improving the deployment efficiency of distributed inference clusters, and facilitating plug-and-play networking.

[0189] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0190] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0191] The following is a detailed reference. Figure 6 This diagram illustrates a suitable structural design for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0192] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0193] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the method for constructing a distributed inference cluster according to embodiments of the present invention.

[0194] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0195] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the method for constructing a distributed inference cluster shown in the above embodiments is implemented.

[0196] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0197] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for constructing a distributed inference cluster, characterized in that, The method includes: Obtain the first external network connectivity status of the first terminal device; Based on the message listening thread of the first terminal device, the link layer broadcast message of the second terminal device is obtained, and the second terminal device is the peer device of the first terminal device; the first terminal device and the second terminal device exchange information based on the link layer broadcast message. Parse the link layer broadcast message to obtain the device information of the second terminal device, the device information including the second external network connectivity status; Based on the first external network connectivity status and the second external network connectivity status, the master device and the slave device are determined from the first terminal device and the second terminal device; In response to the master device and the slave device passing connectivity verification, a distributed inference cluster is formed on the master device and the slave device.

2. The method according to claim 1, characterized in that, The step of parsing the link layer broadcast message to obtain the device information of the second terminal device includes: If the link layer broadcast message is a User Datagram Protocol (UDP) message, then based on the UDP, the UDP message is parsed to obtain the second external network connectivity status of the second terminal device; or, In response to the link layer broadcast message being a Dynamic Host Configuration Protocol (DHCP) discovery message, the additional information field of the DHCP discovery message is parsed based on DHCP to obtain the second external network connectivity status corresponding to the additional information field. or, If the link layer broadcast message is a link layer discovery protocol message, then the type-length-value field of the link layer discovery protocol message is parsed based on the link layer discovery protocol to obtain the second external network connectivity status of the second terminal device.

3. The method according to claim 2, characterized in that, Also includes: Parse the link layer broadcast message to obtain the device name of the second terminal device; the device information also includes the device name. In response to the device name passing the validity verification, the device information of the second terminal device is stored in the neighbor node mapping table corresponding to the first terminal device; If the device name fails the validity check, the link layer broadcast message of the second terminal device is discarded.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the master device and slave device from the first terminal device and the second terminal device based on the first external network connectivity status and the second external network connectivity status includes: Determine whether the first external network connectivity status and the second external network connectivity status are consistent; If the first external network connectivity status and the second external network connectivity status are the same, then the first device identifier corresponding to the first terminal device and the second device identifier corresponding to the second terminal device are obtained respectively. Convert the first device identifier and the second device identifier into a first value and a second value of preset bytes; Based on the comparison between the first value and the second value, the master device and the slave device are determined from the first terminal device and the second terminal device; If the first external network connectivity status indicates that the first terminal device has external network access capability, and the second external network connectivity status indicates that the second terminal device does not have external network access capability, then the first terminal device is determined as the master device, and the second terminal device is determined as the slave device. If the first external network connectivity status indicates that the first terminal device does not have external network access capability, while the second external network connectivity status indicates that the second terminal device has external network access capability, then the second terminal device is determined as the master device, and the first terminal device is determined as the slave device.

5. The method according to claim 4, characterized in that, The connectivity verification of the master device and the slave device includes: Detect whether the master device is configured with the first master address in the target network segment, and whether the slave device is configured with the first slave address in the target network segment; If the master device is configured with the first master address and the slave device is configured with the first slave address, then the connectivity of the master device and the slave device is verified based on the first master address and the first slave address. If the master device is not configured with the first master address, and / or the slave device is not configured with the first slave address, then a preset second master address is configured for the master device, and a preset second slave address is configured for the slave device; Based on the second master address and the second slave address, connectivity verification is performed on the master device and the slave device.

6. The method according to claim 5, characterized in that, Also includes: In response to the existence of multiple slave devices, the third device identifier corresponding to each slave device is obtained; Each of the aforementioned third device identifiers is converted into a third numerical value of preset bytes; According to the sorting result of the third value, a preset second slave address is configured for each of the slave devices in turn.

7. The method according to claim 5, characterized in that, Also includes: In response to the master device and the slave device failing the connectivity verification, the number of connectivity verification failures is obtained; If the number of connectivity verification failures does not exceed a preset value, then the connectivity verification of the master device and the slave device is restarted at a preset interval. If the number of connectivity verification failures exceeds a preset value, the network addresses of the master device and the slave device are reset, and connectivity verification is performed based on the reset network addresses.

8. The method according to claim 1, characterized in that, The process of establishing a distributed inference cluster on the master device and the slave device includes: The master device is controlled to start the head node service of the distributed inference cluster, and the local address of the master device is used to determine the cluster address of the distributed inference cluster. Based on the node information of the distributed inference cluster, a communication channel is constructed between the master device and the slave device, and the inference service of the distributed inference cluster is started. Control the slave device to start the worker node service of the distributed inference cluster, and add the slave device to the distributed inference cluster corresponding to the master device; The slave device is controlled to connect to the master device via the communication channel to wait for scheduling by the master device.

9. A device for building a distributed inference cluster, characterized in that, The device includes: The acquisition module is used to acquire the first external network connectivity status of the first terminal device; The message listening module is used to acquire the link layer broadcast message of the second terminal device based on the message listening thread of the first terminal device, wherein the second terminal device is the peer device of the first terminal device; the first terminal device and the second terminal device exchange information based on the link layer broadcast message. The message parsing module is used to parse the link layer broadcast message and obtain the device information of the second terminal device, including the second external network connectivity status; The master-slave device determination module is used to determine the master device and the slave device from the first terminal device and the second terminal device based on the first external network connectivity status and the second external network connectivity status; A cluster building module is used to build a distributed inference cluster on the master device and the slave device in response to the master device and the slave device passing connectivity verification.

10. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method for constructing a distributed inference cluster as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the method for constructing a distributed inference cluster according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the method for constructing a distributed inference cluster according to any one of claims 1 to 8.

Images