Topology reconfiguration method and device for cooperative computing, cooperative computing system and equipment

Abstract

The application is suitable for the technical field of data processing, and provides a topology reconstruction method and device for collaborative computing, a collaborative computing system and equipment, the method comprising: obtaining topology reconstruction decision information corresponding to a collaborative computing task; determining a target main intelligent chip in a current topology reconstruction process, and determining a target switching boundary according to the topology reconstruction decision information; in the case where the collaborative computing task runs to the target switching boundary, obtaining topology reconstruction reference information of other intelligent chips by the target main intelligent chip, the other intelligent chips being intelligent chips participating in the current topology reconstruction; generating a target logical topology configuration according to the topology reconstruction reference information; and reconstructing a current logical topology formed by the multiple intelligent chips via an exchange interconnection network into a target logical topology based on the target logical topology configuration. The above scheme can improve the collaborative adaptation capability of a multi-chip collaborative computing system in a complex task and a dynamic running environment.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a topology reconstruction method, apparatus, collaborative computing system and device for collaborative computing. Background Technology

[0002] As the scale and application scenarios of artificial intelligence models continue to expand, a single intelligent chip often struggles to meet the demands of complex model inference, image processing, and video processing tasks in terms of computing power, storage capacity, and data throughput. To improve the overall processing capability of the system, current technologies typically connect multiple intelligent chips with artificial intelligence computing capabilities through a switching interconnection network, enabling data transmission between the multiple intelligent chips and allowing them to collaboratively execute the same or one computing task.

[0003] In existing multi-chip collaborative computing systems, multiple intelligent chips typically process data according to pre-defined roles, paths, or collaborative methods. This approach is relatively simple to implement and suitable for scenarios where task types and operating states are relatively stable. However, in actual artificial intelligence computing processes, different tasks or different execution stages of the same task may have different computing and data interaction requirements, and the load, storage usage, temperature, or power consumption of each intelligent chip may also change during operation.

[0004] In the aforementioned scenario, if the collaborative relationships between multiple chips remain fixed over a long period, it becomes difficult to adapt to changes in computational task complexity, task execution status, and chip execution status. This can easily limit the overall collaborative capability of the multi-chip collaborative computing system, thereby affecting the system's resource utilization, computational efficiency, and operational stability. Therefore, improving the collaborative adaptability of multi-chip collaborative computing systems under complex tasks and dynamic operating environments is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a topology reconstruction method, apparatus, collaborative computing system and device for collaborative computing, which can improve the collaborative adaptability of multi-chip collaborative computing systems in complex tasks and dynamic operating environments.

[0006] Firstly, a topology reconfiguration method for collaborative computing is provided, applied to a collaborative computing system including multiple smart chips and a switching interconnection network; the topology reconfiguration method includes: Obtain topology reconstruction decision information corresponding to the collaborative computing task. The topology reconstruction decision information is used to characterize the adjustment requirements of the collaborative computing task for the logical topology and the adaptation status of the multiple smart chips to the topology reconstruction. The target main intelligent chip corresponding to the current topology reconstruction process is determined, and the target switching boundary is determined according to the topology reconstruction decision information, wherein the target switching boundary is used to characterize the task execution position that allows adjustment of the logical topology; When the collaborative computing task reaches the target switching boundary, the topology reconstruction reference information of other intelligent chips is obtained through the target main intelligent chip, and the other intelligent chips are the intelligent chips that participate in this topology reconstruction; Generate the target logical topology configuration based on the topology reconstruction reference information; Based on the target logical topology configuration, the current logical topology formed by the multiple smart chips via the switching interconnection network is reconstructed into the target logical topology.

[0007] In this technical solution, the topology reconstruction decision information corresponding to the collaborative computing task is first obtained. This allows the system to understand the current collaborative computing task's adjustment requirements for the logical topology and the adaptation status of each intelligent chip to the topology reconstruction. Based on this, the target master intelligent chip corresponding to the current topology reconstruction process is determined, and the target switching boundary is identified. This ensures that the topology reconstruction has a corresponding master control and coordination object, and that the adjustment of the logical topology occurs at a suitable task execution position, reducing the risk of data interaction chaos or synchronization anomalies during task execution. Furthermore, when the collaborative computing task reaches the target switching boundary, the topology reconstruction reference information of other intelligent chips is obtained through the target master intelligent chip. This allows the generation process of the target logical topology configuration to incorporate the actual state of the intelligent chips participating in the reconstruction. Subsequently, a target logical topology configuration is generated based on the topology reconstruction reference information, and the current logical topology is reconstructed into the target logical topology based on this configuration. This allows for adjustments to the task allocation relationship, data transmission relationship, and collaborative communication relationship among multiple intelligent chips without changing the physical connection relationship, thereby improving the collaborative computing system's adaptability to task changes and chip operating state changes, and enhancing the resource utilization and collaborative computing efficiency of multiple intelligent chips.

[0008] In one possible implementation, the topology reconfiguration decision information includes at least one of the task execution status of the collaborative computing task and the operating status of the plurality of intelligent chips, wherein determining the target main intelligent chip corresponding to the current topology reconfiguration process includes: If the current topology reconstruction process has a specified initialization master smart chip, the specified initialization master smart chip is determined as the target master smart chip; If the current main intelligent chip meets the conditions for master control migration, the target main intelligent chip is determined from the multiple intelligent chips based on their operating status and / or the task execution status of the collaborative computing task.

[0009] In this implementation, by combining the initialization specification and the master control migration conditions during the operation process to determine the target master intelligent chip, the master control adaptability of the topology reconstruction process can be improved, and the effect of topology reconstruction can be avoided when the fixed master control is not suitable for the current task or chip state.

[0010] In one possible implementation, determining the target switching boundary based on the topology reconstruction decision information includes: In the case where the topology reconstruction scenario is a model stage switching scenario, the model stage boundary is determined as the target switching boundary; In the case where the topology reconstruction scenario is an inference request switching scenario or a task batch switching scenario, the inference request boundary or the task batch boundary is determined as the target switching boundary; When the topology reconstruction scenario is an image region processing scenario, a video frame segment processing scenario, or a time segment processing scenario, the image region boundary, video frame segment boundary, or time segment boundary is determined as the target switching boundary.

[0011] In this implementation, by determining the corresponding target switching boundary for different topology reconstruction scenarios, the logical topology adjustment can be made to occur at a more suitable task execution position, reducing the risk of data interaction anomalies or synchronization anomalies during task execution.

[0012] In one possible implementation, the topology reconfiguration reference information includes status information and topology switching confirmation information; the step of generating a target logical topology configuration based on the topology reconfiguration reference information includes: When the topology switching confirmation information indicates that the other intelligent chips meet the topology switching conditions, the target topology sub-configuration is determined based on the status information and the topology reconstruction scenario corresponding to the collaborative computing task. The target logical topology configuration is generated based on the target topology sub-configuration.

[0013] In this implementation, by combining state information and topology reconstruction scenario to determine the target topology sub-configuration when other smart chips meet the topology switching conditions, the matching degree between the target logical topology configuration and the current task requirements and chip status can be improved.

[0014] In one possible implementation, the target topology sub-configuration includes a parallel topology configuration, which includes at least one of a tensor parallel topology configuration, a pipelined parallel topology configuration, or a hybrid parallel topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: In the case where the parallel topology configuration includes a tensor parallel topology configuration, a smart chipset for performing the same parallel computation object is determined, and the target logical topology configuration is generated based on the smart chipset. In the case where the parallel topology configuration includes a pipelined parallel topology configuration, a smart chipset for executing different pipeline stages is determined, and the target logic topology configuration is generated based on the smart chipset. In the case where the parallel topology configuration includes a hybrid parallel topology configuration, the multiple smart chips are divided into multiple computing groups, the tensor parallel relationships within each computing group and the pipeline connection relationships between different computing groups are determined, and the target logical topology configuration is generated based on the multiple computing groups, the tensor parallel relationships and the pipeline connection relationships. The same parallel computing object includes at least one of the same model layer, the same operator, or the same model stage; the different pipeline stages include at least one of different model levels, different model modules, or different execution stages.

[0015] In this implementation, by generating the target logical topology configuration based on tensor parallelism, pipeline parallelism, or hybrid parallelism, the collaborative relationship between multiple smart chips can be better adapted to the model parallel computing requirements, thereby improving the collaborative computing efficiency during the model inference process.

[0016] In one possible implementation, the target topology sub-configuration includes a task grouping topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: Based on the task scheduling requirements of multiple collaborative computing tasks and the aforementioned status information, the intelligent chipset corresponding to each collaborative computing task is determined. The target logical topology configuration is generated based on the intelligent chipset corresponding to each collaborative computing task; The task scheduling requirements include at least one of the following: task priority, task real-time requirements, or task execution status.

[0017] In this embodiment, by determining the intelligent chipset corresponding to each task based on the task scheduling requirements and status information of multiple collaborative computing tasks, the rationality of resource allocation and task processing efficiency when multiple tasks are executed in parallel can be improved.

[0018] In one possible implementation, the target topology sub-configuration includes a local communication topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: Based on the region or segment division information corresponding to the collaborative computing task and the status information, determine the local point-to-point communication relationship between the smart chips corresponding to the spatial region or time segment. The target logical topology configuration is generated based on the local point-to-point communication relationships; The region or segment division information includes at least one of image region, image block, resolution level, video frame, frame group or time segment.

[0019] In this embodiment, by determining the local point-to-point communication relationship between corresponding chips in a spatial region or time segment based on the region or segment division information, the data interaction efficiency and processing continuity between image regions, video frames, or time segments can be improved.

[0020] Secondly, embodiments of this application provide a collaborative computing topology reconfiguration device, applied to a collaborative computing system including multiple smart chips and a switching interconnection network; the topology reconfiguration device includes: The acquisition module is used to acquire topology reconstruction decision information corresponding to the collaborative computing task. The topology reconstruction decision information is used to characterize the adjustment requirements of the logical topology of the collaborative computing task and the adaptation status of the multiple smart chips to topology reconstruction. The determination module is used to determine the target main intelligent chip corresponding to the current topology reconstruction process, and to determine the target switching boundary based on the topology reconstruction decision information, wherein the target switching boundary is used to characterize the task execution position that allows adjustment of the logical topology; The reference information acquisition module is used to acquire topology reconstruction reference information of other smart chips through the target main smart chip when the collaborative computing task runs to the target switching boundary. The other smart chips are the smart chips that participate in this topology reconstruction. A configuration generation module is used to generate a target logical topology configuration based on the topology reconstruction reference information. The topology reconstruction module is used to reconstruct the current logical topology formed by the multiple smart chips via the switching interconnection network into the target logical topology based on the target logical topology configuration.

[0021] Thirdly, embodiments of this application provide a collaborative computing system, including multiple smart chips and a switching interconnection network, wherein the multiple smart chips are connected through the switching interconnection network; In this embodiment, at least one of the plurality of smart chips is configured to perform the topology reconfiguration method as described in any of the first aspects above.

[0022] Fourthly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the topology reconfiguration method as described in any of the embodiments of the first aspect above.

[0023] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the topology reconfiguration method as described in any embodiment of the first aspect.

[0024] In a sixth aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, enables the implementation of the topology reconfiguration method as described in any embodiment of the first aspect.

[0025] The beneficial effects of the second to sixth aspects mentioned above can be referred to the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the structure of a collaborative computing system provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a collaborative computing topology reconstruction method provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the generation of logical topology configuration based on a model-parallel approach, provided in an embodiment of this application. Figure 4 This is a logical topology diagram of the image region and video frame segment generation task provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of a topology reconfiguration device for collaborative computing provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0028] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0029] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0031] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0033] As the scale and application scenarios of artificial intelligence models continue to expand, a single intelligent chip often struggles to meet the demands of complex model inference, image processing, and video processing tasks in terms of computing power, storage capacity, and data throughput. To improve the overall processing capability of the system, current technologies typically connect multiple intelligent chips with artificial intelligence computing capabilities through a switching interconnection network, enabling data transmission between the multiple intelligent chips and allowing them to collaboratively execute the same or one computing task.

[0034] In existing multi-chip collaborative computing systems, multiple intelligent chips typically process data according to pre-defined roles, paths, or collaborative methods. This approach is relatively simple to implement and suitable for scenarios where task types and operating states are relatively stable. However, in actual artificial intelligence computing processes, different tasks or different execution stages of the same task may have different computing and data interaction requirements, and the load, storage usage, temperature, or power consumption of each intelligent chip may also change during operation.

[0035] In the aforementioned scenario, if the collaborative relationships between multiple chips remain fixed over a long period, it becomes difficult to adapt to changes in computational task complexity, task execution status, and chip execution status. This can easily limit the overall collaborative capability of the multi-chip collaborative computing system, thereby affecting the system's resource utilization, computational efficiency, and operational stability. Therefore, improving the collaborative adaptability of multi-chip collaborative computing systems under complex tasks and dynamic operating environments is an urgent problem to be solved.

[0036] To address the aforementioned issues, this application provides a method, apparatus, system, and device for topology reconfiguration in collaborative computing. The method first acquires topology reconfiguration decision information corresponding to the collaborative computing task, enabling the system to understand the current collaborative computing task's adjustment requirements for the logical topology and the adaptation status of each intelligent chip to the topology reconfiguration. Based on this, the target master intelligent chip corresponding to the current topology reconfiguration process is determined, and the target switching boundary is defined. This ensures that the topology reconfiguration has a corresponding master control and coordination object, and that the adjustment of the logical topology occurs at a suitable task execution location, reducing the risk of data interaction chaos or synchronization anomalies during task execution. Furthermore, during the collaborative computing task operation... When reaching the target switching boundary, the topology reconstruction reference information of other intelligent chips is obtained through the target master intelligent chip. This allows the generation process of the target logical topology configuration to incorporate the actual state of the intelligent chips involved in the reconstruction. Subsequently, the target logical topology configuration is generated based on the topology reconstruction reference information, and the current logical topology is reconstructed into the target logical topology based on this configuration. This allows for adjustments to the task allocation, data transmission, and collaborative communication relationships among multiple intelligent chips without altering their physical connections. This improves the adaptability of the collaborative computing system to changes in task and chip operating states, thereby enhancing the resource utilization and collaborative computing efficiency of multiple intelligent chips.

[0037] The following detailed description of the topology reconstruction method, apparatus, collaborative computing system, electronic device, computer-readable storage medium, and computer program product of collaborative computing in this application is provided with reference to the accompanying drawings.

[0038] See Figure 1 A schematic diagram of a collaborative computing system is shown; for example... Figure 1 As shown, this system can interact with the upper-layer task scheduling module, business application terminal, image acquisition terminal, video processing terminal, or external processing devices to receive collaborative computing tasks and output the corresponding computing results. For ease of explanation, Figure 1 Only the parts relevant to the embodiments of this application are shown.

[0039] like Figure 1As shown, a collaborative computing system can include multiple intelligent chips, a switching interconnect network, and a topology configuration mechanism. Multiple intelligent chips can be connected via the switching interconnect network, which provides a data exchange path between them. Intelligent chips can be chips capable of AI model inference, image or video data processing, intermediate tensor caching, and point-to-point data communication. In actual products, intelligent chips can be AI SoCs, NPUs, ISPs, CIS edge processing chips, or image processing chips with AI acceleration modules. The switching interconnect network can employ PCIe switches, optical interconnect switching networks, high-speed inter-chip interconnect networks, in-package interconnect networks, or other interconnect structures that support configurable data exchange between multiple intelligent chips.

[0040] like Figure 1 As shown, different intelligent chips can undertake different parts of the collaborative computing tasks. For example, some intelligent chips can handle model-stage processing, some can handle tensor segmentation, some can handle context caching, some can handle task branching, and others can handle image region or video frame processing. The target master intelligent chip can generate the target logical topology configuration and configure the collaborative relationships between multiple intelligent chips through a switching interconnection network.

[0041] In this embodiment, the physical connections between multiple smart chips can remain unchanged. The system can adjust the communication objects, data flow directions, task allocation relationships, synchronization relationships, and transmission priorities between the multiple smart chips through a topology configuration mechanism, thereby forming different logical topologies. In other words, the logical topology in this application does not require changing the physical connections between multiple smart chips. Instead, by configuring the data interaction relationships and task collaboration relationships between multiple smart chips, the same collaborative computing system can adapt to different artificial intelligence model inference tasks, image processing tasks, video processing tasks, or scenarios where multiple tasks are executed in parallel.

[0042] In this embodiment, the topology configuration mechanism can be set in the target main intelligent chip, or in the switching network controller, driver layer, firmware, or management module, or it can be implemented collaboratively by the target main intelligent chip and the switching network controller. The target main intelligent chip is used to generate or maintain the logical topology configuration and coordinate the task cooperation and data interaction relationships between multiple intelligent chips.

[0043] like Figure 1As shown, different intelligent chips can undertake different task parts according to the collaborative computing task, such as model stage, tensor segmentation, context caching, task branching, image region or video frame segment, etc. The switching interconnection network can provide data exchange paths between different intelligent chips, enabling multiple intelligent chips to form different data transmission relationships and collaborative computing relationships based on logical topology configuration without changing the physical connection relationship.

[0044] In one possible implementation, the logical topology configuration can exist in the form of a topology configuration table, configuration instructions, or a set of configuration parameters. This configuration indicates the communication objects, task allocation relationships, data flow directions, synchronization relationships, and transmission priorities among multiple intelligent chips. The target master intelligent chip can issue configurations to relevant intelligent chips based on the logical topology configuration, or complete the configuration through the control module corresponding to the switching interconnection network, enabling multiple intelligent chips to form a logical topology that matches the current collaborative computing task.

[0045] It should be noted that, Figure 1 The model stages, tensor slicing, context caching, task branches, image regions, or video frame segments shown in the diagram are merely illustrative examples and do not constitute a limitation on the number of smart chips, chip type, task allocation method, or logical topology. Figure 1 The target main intelligent chip in this context is only used to refer to the intelligent chip that undertakes the function of generating or coordinating topology configuration during the current topology reconstruction process. In other embodiments, the target main intelligent chip can be set or adjusted according to the actual application scenario.

[0046] Based on the aforementioned collaborative computing system, this application further provides a collaborative computing topology reconfiguration method. This method can be executed by the target master intelligent chip in the collaborative computing system, or it can be executed collaboratively by the switching network controller, driver layer, firmware, management module, or the aforementioned components. For ease of explanation, the following mainly uses the execution of this method by the collaborative computing system as an example to illustrate the specific implementation process of the topology reconfiguration method.

[0047] See Figure 2 The diagram illustrates a flowchart of a collaborative computing topology reconstruction method provided in an embodiment of this application; this method can be applied to, for example... Figure 1 The collaborative computing system shown; specifically, such as Figure 1 As shown, the method may include the following steps: Step 201: Obtain the topology reconstruction decision information corresponding to the collaborative computing task.

[0048] Collaborative computing tasks refer to computational tasks that require the coordinated execution of multiple intelligent chips. These collaborative computing tasks can be AI model inference tasks, image processing tasks, video processing tasks, or multiple AI tasks executed in parallel. During execution, collaborative computing tasks may involve model stage processing, tensor segmentation processing, context caching processing, task branching processing, image region processing, or video frame processing, among other tasks.

[0049] The topology reconfiguration decision information is used to characterize the adjustment requirements of the collaborative computing task on the logical topology and the adaptation status of multiple intelligent chips to the topology reconfiguration. This topology reconfiguration decision information can include information related to the collaborative computing task, or information related to multiple intelligent chips, enabling the system to determine whether the current logical topology needs to be adjusted by combining task-side requirements and chip-side status.

[0050] In this application embodiment, logical topology refers to the task collaboration and data interaction relationships formed by multiple intelligent chips while maintaining their physical connections. This logical topology can be reflected in the communication objects, data flow directions, task allocation relationships, synchronization relationships, and transmission priorities between different intelligent chips. In other words, logical topology does not necessarily indicate a change in hardware connections, but rather a change in the collaborative computing method among multiple intelligent chips.

[0051] The need for adjustments to the logical topology in collaborative computing tasks refers to the need to adjust the collaborative relationship between multiple smart chips when changes occur in task type, model parallelism, task execution stage, inference request, task batch, image region, or video frame segment.

[0052] The adaptation status of multiple intelligent chips to topology reconstruction refers to whether the current operating state of multiple intelligent chips is suitable for participating in topology reconstruction. This adaptation status can be determined based on information such as the computing load, storage usage, temperature, power consumption, task progress, intermediate cache status, or communication status of each intelligent chip, and is used to reflect the degree of adaptation of each intelligent chip to continue undertaking the current task, participate in new task allocation, or participate in new data interaction relationships.

[0053] Specifically, after receiving a collaborative computing task from the collaborative computing system, the main intelligent chip can acquire task-side information such as the task type, model parallelism mode, current execution stage, inference request status, task batch status, image region partitioning, or video frame segment partitioning. It can also acquire chip-side information reported by multiple intelligent chips, such as load, storage usage, temperature, power consumption, task progress, or intermediate cache status. Subsequently, the collaborative computing system can use this task-side and chip-side information as topology reconstruction decision information corresponding to the collaborative computing task, or generate corresponding topology reconstruction decision information based on this information.

[0054] For example, in artificial intelligence model inference tasks, multiple intelligent chips can each handle processing tasks such as the model stage, tensor slicing, or context caching. When the model inference task enters a new model stage, or when the currently participating intelligent chip experiences changes in load, temperature, power consumption, or intermediate cache state, the collaborative computing system can acquire information about the model stage changes and the state information of the relevant intelligent chips. This information is then used as topology reconfiguration decision information to characterize the collaborative computing task's need to adjust the logical topology and the adaptation status of the multiple intelligent chips to the topology reconfiguration.

[0055] Step 202: Determine the target main intelligent chip corresponding to the current topology reconstruction process, and determine the target switching boundary based on the topology reconstruction decision information.

[0056] In this embodiment, the current topology reconstruction process refers to the process by which the collaborative computing system prepares to adjust the logical topology between multiple intelligent chips in response to changes in the current collaborative computing task or the current operating state. This process can occur during the execution of the collaborative computing task, or after task switching, stage changes, or chip state changes.

[0057] The target master intelligent chip refers to the intelligent chip used for generating, coordinating, or maintaining the topology configuration during the current topology reconstruction process. The target master intelligent chip is not a fixed physical master control chip, but rather a coordinating object in the current topology reconstruction process. It can be used to organize information interaction between relevant intelligent chips and provide the master control foundation for the subsequent generation of the target logical topology configuration.

[0058] Specifically, after obtaining topology reconfiguration decision information, the collaborative computing system can determine the target main intelligent chip corresponding to the current topology reconfiguration process based on the task execution status and the status of multiple intelligent chips reflected therein.

[0059] For example, in artificial intelligence model inference tasks, multiple intelligent chips can each handle different model stages, tensor slicing, or context caching. When a collaborative computing system determines that the current collaborative relationship needs adjustment based on topology reconstruction decision information, it can first determine the target main intelligent chip corresponding to the current topology reconstruction process.

[0060] In one possible implementation, the topology reconfiguration decision information includes at least one of the task execution status of the collaborative computing task and the operating status of multiple intelligent chips, determining the target main intelligent chip corresponding to the current topology reconfiguration process, including: If the current topology reconstruction process has a specified master smart chip to initialize, the specified master smart chip will be determined as the target master smart chip. If the current main intelligent chip meets the conditions for master control migration, the target main intelligent chip is determined from multiple intelligent chips based on the operating status of multiple intelligent chips and / or the task execution status of collaborative computing tasks.

[0061] In this embodiment, the topology reconfiguration decision information may include at least one of the task execution status of the collaborative computing task and the operating status of multiple intelligent chips. The task execution status can characterize the current task stage, task type, task priority, model parameter distribution, or task switching status of the collaborative computing task; the operating status of the multiple intelligent chips can characterize the computing load, storage usage, temperature, power consumption, communication location, or current task content of each intelligent chip.

[0062] Specifically, in a collaborative computing system, at least one intelligent chip can be configured as the master intelligent chip. The master intelligent chip is used to generate or maintain logical topology configurations, task allocation tables, model parallel configurations, data transmission relationships, and synchronization relationships. Other intelligent chips can periodically or event-triggeredly report their own operating status and interact with the master intelligent chip or adjacent intelligent chips so that the master intelligent chip can understand the current operating status of the collaborative computing system.

[0063] If a primary intelligent chip is specified during the current topology reconstruction process, the collaborative computing system can identify this specified primary intelligent chip as the target primary intelligent chip. In other words, during system startup, task initialization, or default configuration phases, the pre-designated primary intelligent chip can initially handle functions such as topology configuration generation, task allocation, and synchronization coordination.

[0064] If the current master AI chip meets the conditions for master control migration, the collaborative computing system can determine a new target master AI chip from among multiple AI chips based on their operating status and / or the task execution status of the collaborative computing task. Master control migration conditions may include situations such as the current master AI chip being overloaded, its temperature or power consumption approaching its limits, its storage usage being high, an AI task switching, a change in model parameter distribution, or the current master AI chip no longer being suitable for continuing to perform topology configuration coordination functions. The new target master AI chip is not arbitrarily selected but is determined based on the computing load, storage usage, temperature, power consumption, task priority, model parameter distribution, communication location, or degree of correlation with the current task of each AI chip.

[0065] After the new target master intelligent chip is determined, the original master intelligent chip can send at least a portion of its maintained logical topology configuration, task allocation table, model parallel configuration, data transmission relationships, and synchronization relationships to the new target master intelligent chip to achieve master control state handover. The new target master intelligent chip can continue to maintain the current topology configuration based on the received information and generate a new target logical topology configuration during subsequent topology reconstruction.

[0066] In another possible implementation, the new target master intelligent chip may not rely entirely on the configuration transferred from the original master intelligent chip. Instead, it may regenerate or update the logical topology configuration, task allocation table, model parallel configuration, data transmission relationships, and synchronization relationships based on the status information reported by other intelligent chips, task execution status, and topology switch confirmation information. In other words, during master control migration, the original master intelligent chip can transfer its existing configuration to the new target master intelligent chip, the new target master intelligent chip can regenerate the relevant configuration based on the current system state, or the existing configuration can be transferred first, and then the new target master intelligent chip can update it based on the latest state.

[0067] In this way, when the current main intelligent chip is no longer suitable to assume the main control role, the role of the main intelligent chip can be transferred to an intelligent chip that is more suitable for the current task and operating state. This makes the topology reconstruction process less dependent on a fixed main control chip, external SoC, or host computer, and improves the adaptability of the collaborative computing system to topology reconstruction under task changes and chip state changes.

[0068] After determining the target main intelligent chip corresponding to the current topology reconstruction process, the collaborative computing system also needs to determine the execution timing of the logical topology adjustment. Since collaborative computing tasks typically involve model phase transitions, data transmission, intermediate cache usage, and multi-chip synchronization during operation, directly adjusting the logical topology at an inappropriate task execution location may affect the continuous execution of the current task. Therefore, this embodiment further determines the target switching boundary based on topology reconstruction decision information, enabling the subsequent topology reconstruction process to proceed at a relatively suitable task execution location.

[0069] The target switching boundary is used to characterize the task execution location where the logical topology can be adjusted.

[0070] Specifically, the collaborative computing system can determine the suitable location for logical topology adjustment within the collaborative computing task based on the logical topology adjustment requirements reflected in the topology reconfiguration decision information, and designate this location as the target switching boundary. Consequently, the subsequent topology reconfiguration process can be executed at the target switching boundary under the coordination of the target master intelligent chip, thereby reducing the impact on the current task execution process.

[0071] In one possible implementation, determining the target switching boundary based on topology reconstruction decision information includes: In the case where the topology reconstruction scenario is a model stage switching scenario, the model stage boundary is determined as the target switching boundary; In the case where the topology reconstruction scenario is an inference request switching scenario or a task batch switching scenario, the inference request boundary or task batch boundary is determined as the target switching boundary; In the case of topology reconstruction scenarios that are image region processing scenarios, video frame segment processing scenarios, or time segment processing scenarios, the image region boundary, video frame segment boundary, or time segment boundary is determined as the target switching boundary.

[0072] Among them, the topology reconstruction scenario refers to the specific task scenario in which a collaborative computing task triggers logical topology adjustments during its execution. This topology reconstruction scenario can be determined based on topology reconstruction decision information, such as the task type, model execution stage, inference request status, task batch status, image region processing status, video frame processing status, or time segment processing status of the collaborative computing task.

[0073] The target switching boundary refers to the task execution location where the logical topology can be adjusted. Since multiple intelligent chips may have data transmission, intermediate tensor caching, stage synchronization, and task dependencies during collaborative computing, logical topology adjustments should not be made at arbitrary task execution locations, but rather at relatively suitable task boundaries to reduce the risk of data transmission interruptions, synchronization anomalies, or inconsistent intermediate states.

[0074] In this implementation, the collaborative computing system can determine the current topology reconstruction scenario based on the topology reconstruction decision information, and determine the corresponding target switching boundary based on the current topology reconstruction scenario.

[0075] Specifically, in the case of a topology reconstruction scenario that is a model stage switching scenario, it means that the collaborative computing task is about to move from one model stage to another, or the collaborative relationship between different model stages needs to be adjusted. In this case, the model stage boundary can be determined as the target switching boundary.

[0076] In topology reconfiguration scenarios involving inference request switching or task batch switching, it indicates that the current collaborative computing task may involve switching between different inference requests or different task batches. In this case, the inference request boundary or task batch boundary can be defined as the target switching boundary, ensuring that logical topology adjustments occur between requests or batches, reducing the impact on the execution process within a single inference request or task batch.

[0077] In scenarios where topology reconstruction involves image region processing, video frame processing, or time segment processing, it is explained that collaborative computing tasks may involve adjusting the data interaction relationships between image regions, video frames, or time segments. In this case, the boundaries of the image regions, video frames, or time segments can be defined as target switching boundaries to match the logical topology adjustment with the processing boundaries of the image regions, video frames, or time segments.

[0078] For example, in image generation tasks, different intelligent chips can process different image regions separately. When the task enters the boundary fusion stage between adjacent image regions, the image region boundary can be determined as the target switching boundary. In video generation tasks, different intelligent chips can process different video frames separately. When the task enters the cross-frame consistency processing or adjacent frame interaction stage, the video frame boundary or time segment boundary can be determined as the target switching boundary. Through this method, the topology reconstruction process can be matched with the execution progress of the collaborative computing task.

[0079] Step 203: When the collaborative computing task reaches the target switching boundary, obtain the topology reconstruction reference information of other smart chips through the target main smart chip.

[0080] Among them, the other intelligent chips are the intelligent chips that participate in this topology reconstruction. They may include intelligent chips that undertake the model stage, tensor slicing, context caching, task branching, image region or video frame segment processing in the current collaborative computing task.

[0081] The topology reconstruction reference information refers to the reference information used to generate the target logical topology configuration. This topology reconstruction reference information may include the task progress, intermediate cache status, load, temperature, power consumption, communication status, synchronization status, or topology switching confirmation information of other smart chips.

[0082] Specifically, after determining the target switching boundary, the collaborative computing system can continue to monitor the execution progress of the collaborative computing task. When the collaborative computing task reaches the target switching boundary, the target master intelligent chip can send a status acquisition request to other intelligent chips participating in this topology reconfiguration, or receive information reported periodically or by events from other intelligent chips, thereby obtaining topology reconfiguration reference information from other intelligent chips.

[0083] Step 204: Generate the target logical topology configuration based on the topology reconstruction reference information.

[0084] In this embodiment, the target logical topology configuration refers to the configuration content used to instruct multiple intelligent chips to form a target logical topology during subsequent collaborative computing. This target logical topology configuration can be used to characterize the task allocation relationship, data transmission relationship, communication objects, synchronization relationship, and transmission priority among the multiple intelligent chips.

[0085] Specifically, the collaborative computing system can aggregate topology reconstruction reference information through the target master intelligent chip and determine the collaborative relationships among multiple intelligent chips in the target logical topology based on the current state, task execution progress, and synchronization status of the intelligent chips participating in this topology reconstruction. Subsequently, the target master intelligent chip can generate a corresponding target logical topology configuration so that the current logical topology can be reconstructed based on this configuration.

[0086] In one possible implementation, the topology reconfiguration reference information includes status information and topology switching confirmation information; generating a target logical topology configuration based on the topology reconfiguration reference information includes: When the topology switching confirmation information indicates that other intelligent chips meet the topology switching conditions, the target topology sub-configuration is determined based on the status information and the topology reconstruction scenario corresponding to the collaborative computing task. Generate the target logical topology configuration based on the target topology sub-configuration.

[0087] The status information can be used to characterize the operational status of the intelligent chips participating in this topology reconfiguration in the current collaborative computing task, such as task progress, intermediate cache status, currently assigned task fragments, current data transmission relationships, or synchronization status. The topology switching confirmation information can be used to characterize whether the relevant intelligent chips have completed the current stage of processing, whether they have completed the necessary data transmission or synchronization processing, and whether entry into the topology switching process is permitted.

[0088] During model initialization, model loading, or task switching phases, the collaborative computing system can allocate model parameters, weight slices, model layers, model branches, or model stages to different intelligent chips and generate an initial logical topology accordingly. During model inference, different intelligent chips can transmit intermediate tensors, feature maps, KVcache, context cache, image blocks, video frames, or frame fragments based on the current logical topology, and also transmit information such as task status, synchronization signals, stage completion signals, and topology switching confirmation signals.

[0089] Since model parameters or weight shards can typically be allocated during the initialization or task switching phases, the dynamic reconstruction in this embodiment primarily addresses the online adjustment of data flow relationships, task shard relationships, and synchronization relationships during the inference phase. In other words, after the collaborative computing task reaches the target switching boundary, the target master intelligent chip can determine whether the intelligent chips participating in this topology reconstruction meet the topology switching conditions based on the status information and topology switching confirmation information reported by other intelligent chips.

[0090] When the topology switching confirmation information indicates that other intelligent chips meet the topology switching conditions, the target master intelligent chip can determine the target topology sub-configuration based on the status information and the topology reconstruction scenario corresponding to the collaborative computing task. This target topology sub-configuration can be used to characterize the local or overall collaborative relationships that need to be adjusted during the current topology reconstruction process. Subsequently, the target master intelligent chip can generate a target logical topology configuration based on the target topology sub-configuration, enabling subsequent topology reconstruction to be adjusted around the data transmission relationships, task fragmentation relationships, and synchronization relationships during the inference phase.

[0091] In one possible implementation, generating a target logical topology configuration based on a target topology sub-configuration includes: In the case of parallel topology configuration including tensor parallel topology configuration, determine the smart chipset used to perform the same parallel computing object, and generate the target logical topology configuration based on the smart chipset; In the case of parallel topology configuration including pipeline parallel topology configuration, determine the smart chipset used to execute different pipeline stages, and generate the target logical topology configuration based on the smart chipset; In the case of parallel topology configuration including hybrid parallel topology configuration, multiple smart chips are divided into multiple computing groups, the tensor parallel relationships within each computing group and the pipeline connection relationships between different computing groups are determined, and the target logical topology configuration is generated based on multiple computing groups, tensor parallel relationships and pipeline connection relationships. Among them, the same parallel computing object includes at least one of the same model layer, the same operator, or the same model stage; different pipeline stages include at least one of different model levels, different model modules, or different execution stages.

[0092] See Figure 3 This illustration shows a schematic diagram of generating logical topology configurations based on a model-parallel approach, as provided in an embodiment of this application. Figure 3 As shown, when the target topology sub-configuration includes a parallel topology configuration, the corresponding target logical topology configuration can be generated according to the model parallelism method. The parallel topology configuration can include at least one of tensor parallel topology configuration, pipelined parallel topology configuration, or hybrid parallel topology configuration.

[0093] For example, when the parallel topology configuration includes a tensor parallel topology configuration, a smart chipset for executing the same parallel computation object can be determined, and a target logical topology configuration can be generated based on that smart chipset. The same parallel computation object can include at least one of the following: the same model layer, the same operator, or the same model stage. Figure 3 As shown on the left, multiple intelligent chips can respectively store or calculate a portion of the input tensor, weight matrix, intermediate features, or attention-related data, and exchange data point-to-point within the group through an interconnection network, thereby forming a tensor parallel topology suitable for high-frequency transmission of intermediate tensors, feature maps, or context buffers.

[0094] For example, when the parallel topology configuration includes a pipelined parallel topology configuration, a smart chipset for executing different pipeline stages can be determined, and a target logical topology configuration can be generated based on the smart chipset. These different pipeline stages can include at least one of different model levels, different model modules, or different execution stages. Figure 3 As shown in the upper right corner, the model can be divided into different smart chips according to the hierarchy, module or execution stage. Inference data is transferred between different smart chips according to the execution order of the model, thus forming a pipeline parallel topology corresponding to the order of the model stages.

[0095] For example, when the parallel topology configuration includes a hybrid parallel topology configuration, multiple intelligent chips can be divided into multiple computing groups, the tensor parallel relationships within each computing group and the pipeline connection relationships between different computing groups can be determined, and a target logical topology configuration can be generated based on the multiple computing groups, tensor parallel relationships, and pipeline connection relationships. Figure 3 As shown below, each computation group can use tensor parallelism to execute the same model stage, while different computation groups can use pipeline parallelism to connect different model stages, thus forming a hybrid parallel topology of tensor parallelism within groups and pipeline parallelism between groups. Figure 3The number of chips, stages, and grouping methods shown are for illustrative purposes only and do not constitute a limitation on the actual number of smart chips, model stages, or grouping methods.

[0096] In another possible implementation, the target topology sub-configuration includes a task grouping topology configuration; generating a target logical topology configuration based on the target topology sub-configuration includes: Based on the task scheduling requirements and status information of multiple collaborative computing tasks, determine the intelligent chipset corresponding to each collaborative computing task. Generate the target logical topology configuration based on the intelligent chipset corresponding to each collaborative computing task; The task grouping topology configuration is used to characterize the correspondence between multiple collaborative computing tasks and multiple smart chipsets. In other words, when multiple collaborative computing tasks are executed in parallel, the collaborative computing system can determine the smart chipset corresponding to each collaborative computing task based on the task scheduling requirements of different collaborative computing tasks and the status information of multiple smart chips. Task scheduling requirements include at least one of task priority, task real-time requirements, or task execution status.

[0097] For example, when multiple AI tasks are executed simultaneously, some tasks may have higher priority or stronger real-time requirements, while others may be in a waiting, low-priority, or background processing state. The collaborative computing system can combine the status information of each intelligent chip, such as load, storage usage, temperature, power consumption, or task progress, to allocate corresponding intelligent chipsets to different collaborative computing tasks, and generate target logical topology configurations based on the intelligent chipsets corresponding to each collaborative computing task.

[0098] In another possible implementation, the target topology sub-configuration includes a local communication topology configuration; generating a target logical topology configuration based on the target topology sub-configuration includes: Based on the region or segment division information and status information corresponding to the collaborative computing task, determine the local point-to-point communication relationship between the smart chips corresponding to the spatial region or time segment. Generate the target logical topology configuration based on local point-to-point communication relationships; The region or segment division information includes at least one of image region, image block, resolution level, video frame, frame group or time segment.

[0099] The local communication topology configuration is used to characterize the local point-to-point communication relationships between intelligent chips corresponding to spatial regions or time segments. The collaborative computing system can determine the intelligent chips that need to perform local data interaction based on the region or segment division information and state information corresponding to the collaborative computing task, and generate the target logical topology configuration based on the determined local point-to-point communication relationships.

[0100] The region or segment division information may include at least one of the following: image region, image block, resolution level, video frame, frame group, or time segment. See also Figure 4 This diagram illustrates the logical topology of the image region and video frame generation task provided in this application embodiment. In the image generation task, an image can be divided into multiple image regions according to spatial regions, and different intelligent chips can be responsible for processing different image regions respectively. When adjacent image regions need to undergo boundary fusion, global consistency correction, or detail enhancement processing, the local point-to-point communication relationship between the intelligent chips corresponding to the adjacent image regions can be determined, enabling them to exchange boundary features, context buffers, or fusion results.

[0101] like Figure 4 As shown, in video generation tasks, videos can be divided into multiple frame segments according to time frames, and different intelligent chips can be responsible for processing different video frame segments or time segments. When the collaborative computing task enters the cross-frame consistency processing stage, the intelligent chips corresponding to adjacent frame segments can be reconstructed into a local point-to-point communication topology, enabling necessary data interaction between adjacent frame segments to support motion continuity correction, keyframe fusion, or cross-frame consistency processing. Figure 4 The number of image regions, video frames, and smart chips shown are merely examples and do not constitute a limitation on the actual region division method, frame segment division method, or number of smart chips.

[0102] Step 205: Based on the target logical topology configuration, reconstruct the current logical topology formed by multiple smart chips via the switching interconnection network into the target logical topology.

[0103] In this application embodiment, the current logical topology refers to the task collaboration relationship and data interaction relationship formed by multiple smart chips through a switching interconnection network before topology reconstruction; the target logical topology refers to the new task collaboration relationship and data interaction relationship formed after the configuration adjustment according to the target logical topology.

[0104] Specifically, after generating the target logical topology configuration, the collaborative computing system can send the target logical topology configuration to the relevant intelligent chips through the target master intelligent chip, or configure the switching interconnection network through the control module corresponding to the switching interconnection network, so that the communication objects, data flow directions, task allocation relationships, synchronization relationships and transmission priorities between multiple intelligent chips can be adjusted according to the target logical topology configuration.

[0105] In the above method embodiments, after obtaining the topology reconstruction decision information corresponding to the collaborative computing task, the collaborative computing system can first determine the target master intelligent chip in the current topology reconstruction process, and further determine the target switching boundary suitable for logical topology adjustment. When the collaborative computing task reaches the target switching boundary, the target master intelligent chip obtains the topology reconstruction reference information of other intelligent chips participating in this topology reconstruction, and generates a target logical topology configuration accordingly. Subsequently, based on the target logical topology configuration, the collaborative computing system reconstructs the current logical topology formed by multiple intelligent chips through a switching interconnection network into the target logical topology. Thus, without changing the physical connection relationship between multiple intelligent chips, the task allocation relationship, data transmission relationship, and synchronization relationship between multiple intelligent chips can be adjusted according to the collaborative computing task and the chip operating status.

[0106] See Figure 5 The diagram shows a structural schematic of a topology reconfiguration device for collaborative computing provided in an embodiment of this application; for ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0107] Specifically, the collaborative computing topology reconfiguration device 500 can be applied to, for example... Figure 1 The collaborative computing system shown may include a topology reconfiguration device 500 that can include: The acquisition module 501 is used to acquire topology reconstruction decision information corresponding to the collaborative computing task. The topology reconstruction decision information is used to characterize the adjustment requirements of the collaborative computing task for the logical topology and the adaptation status of multiple smart chips to topology reconstruction. The determination module 502 is used to determine the target main intelligent chip corresponding to the current topology reconstruction process, and to determine the target switching boundary based on the topology reconstruction decision information. The target switching boundary is used to characterize the task execution position that allows adjustment of the logical topology. The reference information acquisition module 503 is used to acquire topology reconstruction reference information of other smart chips through the target main smart chip when the collaborative computing task runs to the target switching boundary. The other smart chips are the smart chips that participate in this topology reconstruction. Configuration generation module 504 is used to generate target logical topology configuration based on topology reconstruction reference information; The topology reconstruction module 505 is used to reconstruct the current logical topology formed by multiple smart chips via a switching interconnection network into the target logical topology based on the target logical topology configuration.

[0108] In this embodiment, the acquisition module 501 can be used to perform the step of acquiring topology reconfiguration decision information corresponding to the collaborative computing task in the above method embodiment. Specifically, the topology reconfiguration decision information may include at least one of the task execution status of the collaborative computing task and the operating status of multiple smart chips.

[0109] The determination module 502 may include a main chip initialization determination unit, a main chip migration determination unit, a model stage boundary determination unit, a request batch boundary determination unit, and a region segment boundary determination unit.

[0110] The main chip initialization determination unit is used to determine the specified main smart chip as the target main smart chip when there is a specified main smart chip to be initialized in the current topology reconstruction process.

[0111] The main chip migration determination unit is used to determine the target main intelligent chip from multiple intelligent chips based on the operating status of multiple intelligent chips and / or the task execution status of collaborative computing tasks, provided that the current main intelligent chip meets the main control migration conditions.

[0112] The model stage boundary determination unit is used to determine the model stage boundary as the target switching boundary when the topology reconstruction scenario is a model stage switching scenario.

[0113] The request batch boundary determination unit is used to determine the inference request boundary or task batch boundary as the target switching boundary when the topology reconstruction scenario is an inference request switching scenario or a task batch switching scenario.

[0114] The region segment boundary determination unit is used to determine the image region boundary, video frame segment boundary, or time segment boundary as the target switching boundary when the topology reconstruction scenario is an image region processing scenario, a video frame segment processing scenario, or a time segment processing scenario.

[0115] The reference information acquisition module 503 can be used to perform the step in the above method embodiment of acquiring topology reconstruction reference information of other smart chips through the target master smart chip when the collaborative computing task reaches the target switching boundary. The topology reconstruction reference information may include status information and topology switching confirmation information.

[0116] The configuration generation module 504 may include a topology sub-configuration determination unit, a target configuration generation unit, a tensor parallel configuration generation unit, a pipeline parallel configuration generation unit, a hybrid parallel configuration generation unit, a task grouping configuration generation unit, and a local communication configuration generation unit.

[0117] The topology sub-configuration determination unit is used to determine the target topology sub-configuration based on the status information and the topology reconstruction scenario corresponding to the collaborative computing task, when the topology switching confirmation information indicates that other intelligent chips meet the topology switching conditions.

[0118] The target configuration generation unit is used to generate the target logical topology configuration based on the target topology sub-configuration.

[0119] Tensor parallel configuration generation unit is used to determine the smart chipset for performing the same parallel computing object when the target topology sub-configuration includes a parallel topology configuration and the parallel topology configuration includes a tensor parallel topology configuration, and to generate the target logical topology configuration based on the smart chipset.

[0120] The pipeline parallel configuration generation unit is used to determine the smart chipset for executing different pipeline stages when the target topology sub-configuration includes a parallel topology configuration and the parallel topology configuration includes a pipeline parallel topology configuration, and to generate the target logical topology configuration based on the smart chipset.

[0121] The hybrid parallel configuration generation unit is used to divide multiple smart chips into multiple computing groups when the target topology sub-configuration includes a parallel topology configuration and the parallel topology configuration includes a hybrid parallel topology configuration, determine the tensor parallel relationship within each computing group and the pipeline connection relationship between different computing groups, and generate the target logical topology configuration based on the multiple computing groups, tensor parallel relationship and pipeline connection relationship.

[0122] Among them, the same parallel computing object includes at least one of the same model layer, the same operator, or the same model stage; different pipeline stages include at least one of different model levels, different model modules, or different execution stages.

[0123] The task grouping configuration generation unit is used to determine the intelligent chipset corresponding to each collaborative computing task based on the task scheduling requirements and status information of multiple collaborative computing tasks, when the target topology sub-configuration includes a task grouping topology configuration, and to generate a target logical topology configuration based on the intelligent chipset corresponding to each collaborative computing task. The task scheduling requirements include at least one of task priority, task real-time requirements, or task execution status.

[0124] A local communication configuration generation unit is used to determine the local point-to-point communication relationships between intelligent chips corresponding to spatial regions or time segments based on the region or segment division information and state information corresponding to the collaborative computing task, when the target topology sub-configuration includes a local communication topology configuration, and to generate a target logical topology configuration based on the local point-to-point communication relationships. The region or segment division information includes at least one of image regions, image blocks, resolution levels, video frames, frame groups, or time segments.

[0125] The topology reconstruction module 505 can be used to perform the step of reconstructing the current logical topology formed by multiple smart chips via a switching interconnection network into a target logical topology based on the target logical topology configuration in the above method embodiment.

[0126] It should be noted that each module and unit in the above device embodiments can be used to execute the corresponding steps in the aforementioned method embodiments. For specific implementation processes, please refer to the relevant descriptions in the aforementioned method embodiments, which will not be repeated here.

[0127] See Figure 6 The diagram illustrates the structure of an electronic device according to an embodiment of this application; as shown below. Figure 6 As shown, the electronic device 600 of this embodiment includes: at least one processor 610 ( Figure 6 Only one is shown in the diagram), memory 620, and computer program 621 stored in the memory 620 and executable on the at least one processor 610, wherein the processor 610 executes the computer program 621 to implement the steps in the above-described embodiments of the cooperative computing topology reconfiguration method.

[0128] The electronic device 600 can be a standalone cache management device, server, or intelligent computing device, or it can be a context cache management node, main intelligent chip, or other electronic device component integrated into a multi-intelligent chip collaborative inference system. The electronic device 600 may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will understand that... Figure 6 This is merely an exemplary structure of the electronic device 600 and does not constitute a limitation on the electronic device 600. The electronic device 600 may include more or fewer components than shown, or combine certain components, or use different component connection methods.

[0129] The processor 610 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontroller units (MCUs), image signal processing units, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 610 can be any conventional or special-purpose processing unit that performs control, scheduling, data processing, or interface management functions.

[0130] In some embodiments, the memory 620 may be an internal storage unit of the electronic device 600, such as an on-chip memory, a cache, a register array, or other semiconductor storage unit; in other embodiments, the memory 620 may also be an external storage unit communicatively connected to the electronic device 600. Furthermore, the memory 620 may simultaneously include both internal and external storage units.

[0131] In specific implementations, the processor 610, memory 620, and computer program 621 described in the embodiments of this application can execute the steps in the topology reconstruction method embodiments of the collaborative computing of this application, which will not be repeated here.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / simulator embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0139] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the various method embodiments described above.

[0140] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A topology reconstruction method for collaborative computing, characterized in that, The topology reconfiguration method is applied to a collaborative computing system comprising multiple smart chips and a switching interconnection network; the topology reconfiguration method includes: Obtain topology reconstruction decision information corresponding to the collaborative computing task. The topology reconstruction decision information is used to characterize the adjustment requirements of the collaborative computing task for the logical topology and the adaptation status of the multiple smart chips to the topology reconstruction. The target main intelligent chip corresponding to the current topology reconstruction process is determined, and the target switching boundary is determined according to the topology reconstruction decision information, wherein the target switching boundary is used to characterize the task execution position that allows adjustment of the logical topology; When the collaborative computing task reaches the target switching boundary, the topology reconstruction reference information of other intelligent chips is obtained through the target main intelligent chip, and the other intelligent chips are the intelligent chips that participate in this topology reconstruction; Generate the target logical topology configuration based on the topology reconstruction reference information; Based on the target logical topology configuration, the current logical topology formed by the multiple smart chips via the switching interconnection network is reconstructed into the target logical topology.

2. The topology reconstruction method as described in claim 1, characterized in that, The topology reconfiguration decision information includes at least one of the task execution status of the collaborative computing task and the operating status of the multiple intelligent chips. Determining the target main intelligent chip corresponding to the current topology reconfiguration process includes: If the current topology reconstruction process has a specified initialization master smart chip, the specified initialization master smart chip is determined as the target master smart chip; If the current main intelligent chip meets the conditions for master control migration, the target main intelligent chip is determined from the multiple intelligent chips based on their operating status and / or the task execution status of the collaborative computing task.

3. The topology reconstruction method as described in claim 1, characterized in that, Determining the target switching boundary based on the topology reconstruction decision information includes: In the case where the topology reconstruction scenario is a model stage switching scenario, the model stage boundary is determined as the target switching boundary; In the case where the topology reconstruction scenario is an inference request switching scenario or a task batch switching scenario, the inference request boundary or the task batch boundary is determined as the target switching boundary; When the topology reconstruction scenario is an image region processing scenario, a video frame segment processing scenario, or a time segment processing scenario, the image region boundary, video frame segment boundary, or time segment boundary is determined as the target switching boundary.

4. The topology reconstruction method as described in claim 1, characterized in that, The topology reconfiguration reference information includes status information and topology switching confirmation information; the step of generating the target logical topology configuration based on the topology reconfiguration reference information includes: When the topology switching confirmation information indicates that the other intelligent chips meet the topology switching conditions, the target topology sub-configuration is determined based on the status information and the topology reconstruction scenario corresponding to the collaborative computing task. The target logical topology configuration is generated based on the target topology sub-configuration.

5. The topology reconstruction method as described in claim 4, characterized in that, The target topology sub-configuration includes a parallel topology configuration, which includes at least one of tensor parallel topology configuration, pipelined parallel topology configuration, or hybrid parallel topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: In the case where the parallel topology configuration includes a tensor parallel topology configuration, a smart chipset for performing the same parallel computation object is determined, and the target logical topology configuration is generated based on the smart chipset. In the case where the parallel topology configuration includes a pipelined parallel topology configuration, a smart chipset for executing different pipeline stages is determined, and the target logic topology configuration is generated based on the smart chipset. In the case where the parallel topology configuration includes a hybrid parallel topology configuration, the multiple smart chips are divided into multiple computing groups, the tensor parallel relationships within each computing group and the pipeline connection relationships between different computing groups are determined, and the target logical topology configuration is generated based on the multiple computing groups, the tensor parallel relationships and the pipeline connection relationships. The same parallel computing object includes at least one of the same model layer, the same operator, or the same model stage; the different pipeline stages include at least one of different model levels, different model modules, or different execution stages.

6. The topology reconstruction method as described in claim 4, characterized in that, The target topology sub-configuration includes a task grouping topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: Based on the task scheduling requirements of multiple collaborative computing tasks and the aforementioned status information, the intelligent chipset corresponding to each collaborative computing task is determined. The target logical topology configuration is generated based on the intelligent chipset corresponding to each collaborative computing task; The task scheduling requirements include at least one of the following: task priority, task real-time requirements, or task execution status.

7. The topology reconstruction method as described in claim 4, characterized in that, The target topology sub-configuration includes a local communication topology configuration; generating the target logical topology configuration based on the target topology sub-configuration includes: Based on the region or segment division information corresponding to the collaborative computing task and the status information, determine the local point-to-point communication relationship between the smart chips corresponding to the spatial region or time segment. The target logical topology configuration is generated based on the local point-to-point communication relationships; The region or segment division information includes at least one of image region, image block, resolution level, video frame, frame group or time segment.

8. A topology reconfiguration device for collaborative computing, characterized in that, A collaborative computing system comprising multiple smart chips and a switching interconnection network; the topology reconfiguration device includes: The acquisition module is used to acquire topology reconstruction decision information corresponding to the collaborative computing task. The topology reconstruction decision information is used to characterize the adjustment requirements of the collaborative computing task for the logical topology and the adaptation status of the multiple smart chips to the topology reconstruction. The determination module is used to determine the target main intelligent chip corresponding to the current topology reconstruction process, and to determine the target switching boundary based on the topology reconstruction decision information, wherein the target switching boundary is used to characterize the task execution position that allows adjustment of the logical topology; The reference information acquisition module is used to acquire topology reconstruction reference information of other smart chips through the target main smart chip when the collaborative computing task runs to the target switching boundary. The other smart chips are the smart chips that participate in this topology reconstruction. A configuration generation module is used to generate a target logical topology configuration based on the topology reconstruction reference information. The topology reconstruction module is used to reconstruct the current logical topology formed by the multiple smart chips via the switching interconnection network into the target logical topology based on the target logical topology configuration.

9. A collaborative computing system, characterized in that, It includes multiple smart chips and a switching interconnection network, wherein the multiple smart chips are connected through the switching interconnection network; Wherein, at least one of the plurality of smart chips is configured to perform the topology reconfiguration method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the topology reconfiguration method as described in any one of claims 1 to 7.

Images