Inference task execution method, system, and device

Abstract

The present application relates to the technical field of optical fiber communications, and provides an inference task execution method, system, and device. The method is performed by a first optical access device in an optical fiber access network, and comprises: acquiring an inference task to be executed; and acquiring an execution result of the inference task, wherein the execution result is obtained by executing the inference task via a plurality of target access devices respectively running different sub-modules, and the target access devices comprise other optical access devices capable of communicating with the first optical access device. The method reduces the inference time for executing an inference task, improves inference efficiency, and enhances resource utilization of optical access devices.

Description

A method, system, and device for performing a reasoning task.

[0001] This application claims priority to Chinese Patent Application No. 202411437766.6, filed on October 15, 2024, entitled "A Method, System and Device for Executing a Reasoning Task", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of optical fiber communication technology, and in particular to a method, system and device for performing inference tasks. Background Technology

[0003] With the rapid development of the internet and the popularization of IoT technology, fiber optic access networks, as a high-speed, high-bandwidth access method, occupy an important position in modern communication networks. Fiber optic access networks not only support traditional high-speed internet access but are also gradually taking on the responsibility of supporting various intelligent services, such as intelligent Q&A and voice assistants. These intelligent services often have high demands for computing resources; however, the equipment in fiber optic access networks currently cannot meet the computing resource requirements of these intelligent services. Summary of the Invention

[0004] This application provides a method, system, and device for executing inference tasks, which shortens the inference time, improves inference efficiency, and increases the resource utilization of optical access devices.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, this application provides a method for executing an inference task, which can be executed by a first optical access device in an optical fiber access network. Here, the inference task refers to a task that requires the use of a large model (such as a large language model, a diffusion model, etc.). The first optical access device is a side-side device, such as an optical line terminal (OLT). The method includes: firstly, obtaining the inference task to be executed (such as an intelligent question-answering task or an image generation task); then, obtaining the execution result of the inference task, which is obtained by multiple target access devices running different sub-modules of a preset inference model to execute the inference task; wherein, the target access devices include other optical access devices (also referred to as end-side devices) capable of communicating with the first optical access device, such as optical gateway devices.

[0007] As can be seen from the above, this application breaks down the complex pre-defined inference model into multiple sub-modules and distributes them to target access devices in the fiber optic access network. Each target access device is only responsible for a portion of the computation in the pre-defined inference model. On the one hand, it eliminates the need to send the inference task to the cloud-side device via a long link, making full use of the computing resources of the optical access devices while avoiding long-distance end-edge-cloud transmission, shortening the execution time of the entire inference task, and improving inference efficiency. On the other hand, different optical access devices may have different computing resources. By reasonably allocating the sub-modules in the pre-defined inference model, the computing resources in the fiber optic access network can be utilized more effectively, which helps to achieve load balancing and avoid overload or idleness of optical access devices. Furthermore, since the pre-defined inference model is broken down into multiple sub-modules and runs on different optical access devices, even if one optical access device fails, it will not cause the entire inference task to fail. Other optical access devices can continue to execute the sub-modules corresponding to the failed device, which can flexibly respond to failure crises and improve the reliability of the inference task.

[0008] In one possible implementation, obtaining the inference task to be executed includes: receiving an inference task request sent by a gateway device; the gateway device is any gateway device that accesses the target network through a first optical access device; the inference task request is used to request the execution of the inference task; the method further includes: returning the execution result of the inference task to the gateway device.

[0009] In one possible implementation, the target access device is an optical gateway device that accesses the target network through a first optical access device.

[0010] In this implementation, the computing resources of the optical access devices in the fiber optic access network are fully utilized, enabling the optical access devices to run some sub-modules in the preset inference model, thereby realizing distributed computing of the inference task.

[0011] In one possible implementation, the first optical access device is an optical line terminal (OLT).

[0012] In this implementation, the OLT device acts as an intermediary in the fiber optic access network, connecting end-side devices (such as optical gateway devices) with the target network (such as a local area network or a wide area network). As a core device in the fiber optic access network, the OLT device possesses substantial computing resources. Therefore, it can dynamically allocate sub-modules in the preset inference model based on the operational status of the optical access devices in the connected fiber optic access network, ensuring the smooth execution of inference tasks.

[0013] In one possible implementation, obtaining the execution result of the inference task includes: identifying multiple target access devices and target sub-modules that each target access device needs to run from the optical access devices in the optical fiber access network; executing the inference task by instructing each target access device to run its corresponding target sub-module, and obtaining the execution result.

[0014] In this implementation, the first optical access device breaks down the preset inference model into multiple target sub-modules, determines the target sub-modules to be run by the target access devices, and schedules each target access device to run the corresponding target sub-module. The sub-modules in the preset inference model are flexibly allocated according to the actual situation of the optical access devices, and the target access devices and their corresponding target sub-modules are dynamically adjusted to achieve load balancing, avoid overload or idleness of some optical access devices, and improve the resource utilization efficiency of the optical access devices.

[0015] In one possible implementation, multiple target access devices and target sub-modules that each target access device needs to run are determined from the optical access devices in the optical fiber access network. This includes: obtaining target operation information of the optical access devices, which is used to indicate the model operation capabilities of the optical access devices; and determining multiple target access devices and target sub-modules that each target access device needs to run based on the model operation capabilities of the optical access devices, wherein the model operation capabilities of the target access devices match the capability requirements of the target sub-modules corresponding to the target access devices.

[0016] In this implementation, by acquiring the target operational information of the optical access devices, the first optical access device can accurately understand the model operational capabilities of each optical access device. This allows the first optical access device to allocate sub-modules according to actual capability requirements, ensuring that each target access device can run sub-modules that match its capabilities, thereby avoiding wasted computing resources or overload, and ensuring that inference tasks can be executed sequentially.

[0017] In one possible implementation, the inference task is executed by instructing each target access device to run its corresponding target sub-module to obtain the execution result. This includes: determining the time order in which the target sub-modules run among the target access devices based on the connection relationship of the target sub-modules to be run by each target access device in the preset inference model; sending task execution instructions to each target access device in chronological order, the task execution instructions carrying input information, and the task execution instructions being used to instruct the target access devices to process the input information by running the corresponding target sub-modules; wherein, the input information of the sub-modules on the later target access devices is determined based on the output information of the sub-modules on the earlier target access devices.

[0018] In this implementation, by clearly defining the time sequence of the target sub-modules running among the target access devices, it is possible to ensure that the preset logic and process in the preset inference model can be correctly executed by each target access device, thereby avoiding the failure of the inference task due to the disordered execution order of the target sub-modules and improving the accuracy and reliability of the inference task execution.

[0019] In one possible implementation, the inference task is executed by instructing each target access device to run its corresponding target sub-module to obtain the execution result. The method also includes: obtaining the output information of the sub-module returned by the target access device; and taking the output information of the last sub-module in the preset inference model as the execution result of the inference task.

[0020] In this implementation, the output of each submodule serves as the input for subsequent submodules. Therefore, any data loss or error may affect the accuracy of the final execution result. The first optical access device is used as a scheduling device, responsible for acquiring and managing the output information of each submodule. Based on the connection relationship of the submodules in the preset inference model, the first optical access device can orderly transmit the output information to the target optical access device corresponding to the subsequent submodules, effectively reducing data loss or errors during transmission, ensuring data integrity and accuracy, thereby avoiding the impact of data problems on the final execution result and enhancing the reliability of the inference task.

[0021] In one possible implementation, the target runtime information includes at least one of memory data, computing power data, and central processing unit (CPU) data; the memory data is used to characterize the memory usage in the optical access device, the computing power data is used to characterize the computing power of the optical access device, and the CPU data is used to characterize the CPU usage in the optical access device.

[0022] In this implementation, understanding the memory, computing power, and CPU usage of the optical access devices helps the first optical access device to allocate inference tasks more rationally. For example, tasks requiring a large amount of computing resources can be preferentially assigned to optical access devices with stronger computing power; tasks requiring fast response can be preferentially assigned to optical access devices with lower CPU utilization; and tasks requiring a large amount of storage space can be preferentially assigned to optical access devices with larger memory. This optimized task allocation strategy helps improve the efficiency and accuracy of task execution.

[0023] In one possible implementation, the preset inference model includes multiple first-level sub-modules connected in sequence; each first-level sub-module includes one or more second-level sub-modules, and the multiple second-level sub-modules are connected in parallel; the target sub-module operated by each target access device includes at least one first-level sub-module and / or at least one second-level sub-module.

[0024] In this implementation, on the one hand, the parallel connections between multiple second-level submodules within each first-level submodule allow them to process data simultaneously, significantly improving the parallel processing capability for inference tasks. Especially when dealing with large-scale datasets or complex inference tasks, parallel processing can significantly reduce overall processing time and improve the response speed and throughput of the inference task. On the other hand, by dividing the inference model into multiple first-level submodules, each of which can contain multiple parallel second-level submodules, this modular design offers high flexibility and scalability. When the complexity of the inference task needs to be adjusted or new functionality added, first-level or second-level submodules can be easily added, deleted, or replaced without major modifications to the entire pre-defined inference model. Furthermore, the modular design enhances fault isolation and tolerance; if a first-level or second-level submodule fails, the problem can be quickly located and the faulty module isolated without affecting the normal operation of other submodules.

[0025] In one possible implementation, the multiple target access devices include a first device, a second device, and a third device; wherein the first device and the second device are used to run their respective target sub-modules in parallel; and the first device and the third device are used to run their respective target sub-modules in serial order.

[0026] In this implementation, by running the target sub-modules on the first and second devices in parallel, the processing capabilities of multiple optical access devices can be fully utilized to process different parts of the inference task simultaneously, significantly reducing the total execution time of the inference task and improving the processing speed.

[0027] In one possible implementation, the input information of the target submodule on the first device and the second device is the same; the input of the target submodule on the third device is determined based on the output of the target submodule on the first device and the second device.

[0028] In one possible implementation, the first device, the second device, and the third device are used to run different secondary sub-modules; and the secondary sub-modules run by the first device and the second device belong to the same primary sub-module, while the secondary sub-modules run by the first device and the third device belong to different primary sub-modules.

[0029] In one possible implementation, the method further includes: receiving configuration information; the configuration information includes device identifiers of multiple target access devices and the number of sub-modules in a preset inference model; the device identifiers are used to indicate optical access devices in the optical fiber access network; the number of sub-modules is used to indicate the total number of sub-modules in the preset inference model.

[0030] In this implementation, users can determine the configuration information according to different application scenarios and requirements, and send the configuration information to the first optical access device. Based on the configuration information, the first optical access device can quickly and flexibly configure the target access device and the corresponding sub-modules, shorten the deployment time of the first optical access device for the target access device and the corresponding sub-modules, simplify the deployment process of the first optical access device, and achieve rapid deployment to complete the inference task.

[0031] Secondly, a method for executing a reasoning task is provided. The method is executed by a second optical access device in an optical fiber access network. The method includes: obtaining a task execution instruction; the task execution instruction instructs the processing of input information by running a target submodule in a preset reasoning model; in response to the task execution instruction, running the target submodule and obtaining the output information of the target submodule; the output information is the calculation result of the target submodule on the input information.

[0032] In one possible implementation, obtaining the task execution instruction includes: receiving the task execution instruction sent by the first optical access device; the method further includes: sending the output information of the target submodule to the first optical access device; and the second optical access device accessing the target network through the first optical access device.

[0033] Thirdly, an execution system for an inference task is provided. This system includes a first optical access device and multiple second optical access devices. The second optical access devices access a target network through the first optical access device. The first optical access device is used to acquire the inference task to be executed and send task execution instructions to the multiple second access devices. The task execution instructions instruct the second access devices to process the input information by running the corresponding target submodule in a preset inference model. The second optical access devices are used to receive the task execution instructions and, in response to the task execution instructions, run the target submodule to obtain the output information of the target submodule and return the output information to the first optical access device.

[0034] In this application, by breaking down a complex pre-defined inference model into multiple sub-modules and assigning them to target access devices in the fiber optic access network, each target access device is only responsible for a portion of the computation within the pre-defined inference model. Firstly, this eliminates the need to send the inference task to cloud-side devices via long links, fully utilizing the computing resources of the optical access devices while avoiding long end-edge-cloud transmission, thus shortening the execution time of the entire inference task and improving inference efficiency. Secondly, different optical access devices may have different computing resources. By rationally allocating the sub-modules in the pre-defined inference model, the computing resources in the fiber optic access network can be utilized more effectively, helping to achieve load balancing and preventing overload or idleness of optical access devices. Thirdly, since the pre-defined inference model is broken down into multiple sub-modules running on different optical access devices, even if one optical access device fails, the entire inference task will not fail. Other optical access devices can continue to execute the sub-module corresponding to the failed device, enabling flexible response to failure crises and improving the reliability of the inference task.

[0035] Fourthly, a computing device is provided, comprising: a processor and a memory, the processor being connected to the memory. The memory is used to store computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory, thereby implementing any of the methods provided in the first aspect.

[0036] Fifthly, a chip is provided, comprising: a processor and an interface circuit; the interface circuit is used to receive code instructions and transmit them to the processor; the processor is used to execute the code instructions to perform any of the methods provided in the first aspect above.

[0037] In a sixth aspect, a computer-readable storage medium is provided, storing computer-executable instructions that, when executed on a computer, cause the computer to perform any of the methods provided in the first aspect above.

[0038] In a seventh aspect, a computer program product is provided, including computer execution instructions that, when executed on a computer, cause the computer to perform any of the methods provided in the first aspect above.

[0039] The technical effects of any of the implementation methods in aspects two through seven can be found in the technical effects of different implementation methods in aspect one, and will not be repeated here. Attached Figure Description

[0040] Figure 1 is a schematic diagram of a scenario for implementing model inference based on cloud-side devices according to an embodiment of this application;

[0041] Figure 2 is a system architecture diagram of an execution system for a reasoning task provided in an embodiment of this application;

[0042] Figure 3 is a system architecture diagram of another inference task execution system provided in an embodiment of this application;

[0043] Figure 4 is a flowchart of a method for executing a reasoning task according to an embodiment of this application;

[0044] Figure 5 is a schematic diagram of the connection relationship of a target submodule in a preset inference model provided by an embodiment of this application;

[0045] Figure 6 is a schematic diagram of the connection relationship of another target sub-module in a preset inference model provided by an embodiment of this application;

[0046] Figure 7 is a schematic diagram of the connection relationship of another target sub-module in the preset inference model provided in the embodiment of this application;

[0047] Figure 8 is a schematic diagram of a scenario in which a first optical access device sends a task execution instruction and receives output information from each target access device, according to an embodiment of this application.

[0048] Figure 9 is a schematic diagram of a scenario in which a first optical access device sends a task execution instruction and receives output information from each target access device, according to an embodiment of this application.

[0049] Figure 10 is a schematic diagram of the structure of the preset inference model provided in the embodiment of this application, which includes multiple sub-modules;

[0050] Figure 11 is a schematic diagram showing the relationship between the first device, the second device, and the third device provided in the embodiments of this application;

[0051] Figure 12 is a flowchart illustrating how a first optical access device determines multiple target access devices according to an embodiment of this application.

[0052] Figure 13 is a schematic diagram of a message format of the OMCI protocol provided in an embodiment of this application;

[0053] Figure 14 is a schematic diagram of the field composition of a message type provided in an embodiment of this application;

[0054] Figure 15 is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0056] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.

[0057] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0058] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0059] The following is a description of the terminology used in this application.

[0060] Fiber to the Room (FTTR) is a fiber optic access technology that applies fiber optic networks within buildings such as homes. FTTR extends the fiber optic network directly to every room in a building (e.g., a home), leveraging the data transmission capabilities of fiber optics to achieve high-speed, stable network coverage throughout the entire building. In this application, optical line terminals (OLTs), optical gateway devices (such as optical network units (ONUs), optical network terminals (ONTs), and FTTR gateway devices) are key components of the FTTR network architecture.

[0061] Computing devices (such as cloud servers) leverage large-model technology to deliver efficient intelligent services (such as intelligent question answering and voice assistants) to users. Specifically, computing devices run large and complex models, such as large language models and diffusion models, to perform various inference tasks, thereby providing users with accurate and intelligent personalized services. Meanwhile, with the rapid development of internet technology and the widespread application of the Internet of Things (IoT), fiber optic access networks, with their high speed and high bandwidth, occupy an important position in communication networks. Fiber optic access networks not only enable ultra-high-speed internet access but are also increasingly becoming the foundational support for diverse intelligent services. For example, the aforementioned large models can be deployed on cloud-side devices. Terminal devices request inference tasks from cloud-side devices through optical access devices in the fiber optic access network. The cloud-side devices run the large models, perform the inference tasks, obtain the inference results, and then return the results to the terminal devices via the optical access devices, thus providing the aforementioned intelligent services to users.

[0062] Figure 1 illustrates a scenario of model inference based on cloud-side devices. The end-side device in the fiber optic access network (the FTTR gateway device in Figure 1) receives an inference task sent by a terminal device in the home fiber optic network. This task is then forwarded to the cloud-side device (the server in Figure 1) via an edge-side device (the optical line terminal in Figure 1) and a transmission device (the switch in Figure 1) in the optical transmission network. After executing the inference task, the cloud-side device returns the execution result to the end-side device via the transmission device and the edge-side device. In this scenario, each inference task sent by the terminal device needs to pass through the end-side device, the edge-side device, and the transmission device before finally reaching the cloud-side device. The cloud-side device runs a large model to obtain the inference result, which is then transmitted back to the end-side device via the transmission device and the edge-side device, and finally fed back to the terminal device. This long end-edge-cloud transmission process results in a long execution time and low efficiency for the entire inference task. Furthermore, if the cloud-side device malfunctions, the entire inference task will be difficult to execute successfully.

[0063] In view of this, embodiments of this application provide a method for executing inference tasks. This method involves breaking down a preset inference model, such as a large language model or a diffusion model, into multiple sub-modules. Multiple optical access devices in the fiber optic access network then run their respective sub-modules to obtain the execution result of the inference task. This achieves model inference within the fiber optic access network, fully utilizing the resources of the optical access devices, and solves the problems of long inference time and low efficiency caused by long-link communication when executing inference tasks using cloud-side devices.

[0064] In some embodiments, the inference task execution method provided in this application is executed by a first optical access device in an optical fiber access network. Here, the inference task refers to a task that requires the use of an inference model, also known as a large model (such as a large language model, diffusion model, etc.). The first optical access device is a side-side device, such as an optical line terminal (OLT), a multi-service access network (MSAN), etc. The method includes: first, obtaining the inference task to be executed (such as intelligent question answering, voice assistant, etc.). Then, obtaining the execution result of the inference task. The execution result is obtained by multiple target access devices running different sub-modules of a preset inference model to execute the inference task; the target access devices include other optical access devices (also known as end-side devices) capable of communicating with the first optical access device, such as optical gateway devices.

[0065] As can be seen from the above, this application breaks down a complex pre-defined inference model into multiple sub-modules, and executes the inference task by running different sub-modules of the pre-defined inference model through multiple target access devices. Each target access device is only responsible for a portion of the computation in the pre-defined inference model. Firstly, it eliminates the need to send the inference task to the cloud-side device via a long link, fully utilizing the computing resources of the optical access devices while avoiding long-distance end-edge-cloud transmission, thus shortening the execution time of the entire inference task and improving execution efficiency. Secondly, different optical access devices may have different computing resources. By rationally allocating the sub-modules in the pre-defined inference model, the computing resources in the optical fiber access network can be utilized more effectively, helping to achieve load balancing and preventing overload or idleness of optical access devices. Thirdly, because the pre-defined inference model is broken down into multiple sub-modules and run on different optical access devices, even if one optical access device fails, the entire inference task will not fail. Other optical access devices can continue to run the sub-modules originally run by the failed device, enabling flexible response to failure crises and improving the reliability of the inference task.

[0066] The system architecture of the embodiments of this application will be described below by way of example.

[0067] This application provides an execution system for an inference task. Figure 2 is a system architecture diagram of an execution system for an inference task provided in this application. The execution system for the inference task includes a first optical access device and multiple second optical access devices (target access device 1, target access device 2...target access device n in Figure 2). The first optical access device can also be called a side-side device, such as an OLT device, and the second optical access devices can also be called end-side devices, such as an optical gateway device.

[0068] The first optical access device is used to acquire the inference task to be executed; it sends task execution instructions to multiple target access devices, which instruct the target access devices to process the input information by running the corresponding target sub-module in the preset inference model.

[0069] The target access device is used to receive task execution instructions and, in response to the task execution instructions, run the target submodule and obtain the output information of the target submodule; and return the output information (i.e. the calculation result of the target submodule on the input information) to the first optical access device.

[0070] The target access device is the optical access device selected by the first optical access device from a plurality of second optical access devices.

[0071] In one possible implementation, the first optical access device obtains the inference task to be executed by receiving an inference task request sent by the gateway device, as shown in Figure 2. The inference task request is used to request the execution of the inference task.

[0072] The aforementioned gateway device can be any gateway device that accesses the target network (such as a local area network, wide area network, etc.) through the first optical access device. For example, the gateway device can be an ONU device, an ONT device, or an FTTR gateway device. Here, the FTTR gateway device mainly refers to the FTTR master gateway device; however, theoretically, it can also be an FTTR slave gateway device.

[0073] Optionally, the user can send an inference task request to the gateway device via user instructions, and the gateway device can forward the inference task request to the first optical access device.

[0074] For example, user commands can be voice commands, text commands, gesture commands, etc. The gateway device includes input devices, such as a touchscreen display, microphone, keyboard, etc., through which the user sends inference task requests to the gateway device. For instance, the user performs touch or key operations on the display to send an inference task request to the gateway device. Alternatively, the user inputs a voice command through the microphone to send the inference task request to the gateway device.

[0075] Optionally, users can also input the above user instructions on a terminal device (such as a mobile phone, computer, etc.), and the terminal device can send an inference task request to the first optical access device through the gateway device.

[0076] In another possible implementation, the first optical access device can also receive inference task requests from other devices and obtain inference tasks to be executed. These other devices can be side-side devices (such as OLT devices), devices in the metropolitan area network (such as routers and switches), devices in the core network (such as servers), etc., in addition to the first optical gateway device.

[0077] In another possible implementation, the user can also directly send an inference task request to the first optical access device, instructing the first optical access device to perform an inference task (such as a network diagnostic task). For example, the first optical access device includes input devices, such as a touchscreen display, microphone, keyboard, etc., and the user sends the inference task request to the first gateway device by operating the input devices within the first optical access device.

[0078] In one possible implementation, the first optical access device first identifies multiple target access devices and the target sub-modules that each target access device needs to run from among the optical access devices in the optical fiber access network. Then, it instructs each target optical access device to run its corresponding target sub-module to perform inference tasks, thereby obtaining the execution result. In this way, the first optical access device breaks down the preset inference model into multiple target sub-modules, determines the target sub-modules to be run by the target access devices, and schedules each target access device to run the corresponding target sub-module. It flexibly allocates the sub-modules in the preset inference model according to the actual situation of the optical access devices, dynamically adjusts the target access devices and their corresponding target sub-modules, achieves load balancing, avoids overload or idleness of some optical access devices, and improves the resource utilization efficiency of the optical access devices.

[0079] Optionally, the first optical access device may divide the preset inference model into multiple sub-modules according to the specific internal structure of the preset inference model. The sub-modules may be connected in series or in parallel. The division of sub-modules in the preset inference model will be described in detail in Part II of the subsequent embodiments, and will not be repeated here.

[0080] Optionally, the first optical access device determines the target access device and the target sub-modules that each target access device needs to run, based on the target operation information of the optical access devices in the optical fiber access network (used to characterize the model operation capability of the optical access devices).

[0081] For example, target operational information may include memory data to reflect the memory usage in the optical access device. The first optical access device selects a target access device in the optical fiber access network based on the memory data of the optical access device. Alternatively, target operational information may include computing power data to reflect the computing power of the optical access device. The first optical access device selects a target access device in the optical access network based on the computing power data of the optical access device. Furthermore, target operational information may also include central processing unit (CPU) data. The first optical access device selects each target access device based on the CPU usage represented by the CPU data.

[0082] How the first optical access device selects multiple target access devices in the optical fiber access network, and how it instructs the target access devices to run target submodules, will be described in Part 3 and Part 1 of the subsequent embodiments, respectively, and will not be repeated here.

[0083] In another possible implementation, where optical access devices in the fiber optic access network can communicate with each other, after the first optical access device determines multiple target access devices and their respective target sub-modules from the fiber optic access network, the first optical access device first sends the inference task and the device module corresponding information containing the target sub-modules of each target access device to the target access device corresponding to the first sub-module in the preset inference model, and then receives the output information of the target access device corresponding to the last sub-module, and returns the output information of the target access device corresponding to the last sub-module as the execution result to the gateway device.

[0084] Figure 3 shows the system architecture of another inference task execution system. The first optical access device sends the inference task and device module information (containing the corresponding target sub-modules of each target access device) to the target access device 1 corresponding to the first sub-module (target sub-module 1) in the preset inference model. After running target sub-module 1, target access device 1 directly transmits output information 1 and the corresponding device module information to the target access device 2 corresponding to the second sub-module (target sub-module 2) in the preset inference model. Target access device 2 runs the corresponding target sub-module 2 based on the device module information, obtaining output information 2. This process continues until the output information n of the target access device n corresponding to the last sub-module (target sub-module n) in the preset inference model is obtained. Target access device n then returns the output information n to the first optical access device. This eliminates the need for data transmission between the second and first optical access devices, further shortening the execution time of the inference task.

[0085] In one possible implementation, the first optical access device can be an OLT device, an MSAN device, etc., without specific limitations.

[0086] For example, the first optical access device includes a processor and a memory.

[0087] The processor is connected to the memory, which stores computer execution instructions. The processor executes the computer execution instructions stored in the memory, thereby realizing the above-mentioned reasoning task execution method.

[0088] In one possible implementation, the second optical access device stores a preset inference model. After receiving the task execution instruction from the first optical access device, the second optical access device, as the target access device, runs the corresponding target sub-module.

[0089] In one possible implementation, the second optical access device is an optical gateway device that accesses the target network (such as a local area network, wide area network, etc.) through the first optical access device.

[0090] Optionally, the second optical access device can be an ONU device, an ONT device, an FTTR gateway device, etc., without specific limitations.

[0091] For example, the second optical access device includes a processor, a memory, and a computing unit.

[0092] The processor is connected to the memory, which stores computer execution instructions. The processor executes these instructions to achieve the aforementioned inference task execution method. The computing unit of the second optical access device runs the corresponding target submodule to obtain its output information.

[0093] The memory of the second optical access device can be video memory, or it can be Flash memory, non-volatile random access memory, etc., without any limitation here.

[0094] The computing unit in the second optical access device can be a neural processing unit (NPU), a central processing unit (CPU), a graphics processing unit (GPU), etc.

[0095] In another possible implementation, the second optical access device can also be another optical access device capable of communicating with the first optical access device, such as another OLT device. The first optical access device performs model inference in the fiber optic access network by scheduling other optical access devices.

[0096] Of course, the first optical access device can also be the second optical access device. That is to say, in the process of executing the inference task, in addition to executing their respective sub-modules as optical access devices in the scheduled optical fiber access network, the first optical access device can also be used to execute sub-modules in the preset inference model.

[0097] In one possible implementation, the first optical access device communicates with each of the second optical access devices via the Optical Network Unit Management and Control Interface (OMCI) protocol.

[0098] Optionally, the first optical access device can obtain the model operation capabilities of the second optical access device through the OMCI protocol, instruct the second optical access device to run its respective sub-modules, and transmit the input and output information of the second optical access device, etc.

[0099] It should be noted that the embodiments of this application do not limit the device form of the first optical access device and the second optical access device. In the following embodiments, the system architecture of the inference task execution system provided by the embodiments of this application is described using the example of the first optical access device being an OLT device and the second optical access device being an FTTR gateway device.

[0100] It should be noted that the system architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0101] The following embodiments of this application will be divided into four parts to exemplarily describe the scheme of the execution method of the reasoning task.

[0102] The first part, in conjunction with Figures 4 to 9, introduces the execution method of the reasoning task provided in the embodiments of this application, aiming to introduce the overall process of the execution method of the reasoning task.

[0103] The second part, in conjunction with Figures 10 and 11, introduces the execution method of the reasoning task provided in the embodiments of this application, aiming to show how to split the preset reasoning model into multiple sub-modules.

[0104] The third part, in conjunction with Figure 12, introduces the execution method of the inference task provided in the embodiments of this application, which aims to introduce the specific process by which the first optical access device determines multiple target access devices in the optical fiber access network.

[0105] Part Four, in conjunction with Figures 13 and 14, introduces the execution method of the inference task provided in the embodiments of this application, aiming to introduce the specific implementation of communication between the first optical access device and each target access device.

[0106] In some embodiments, a preset inference model is split into multiple sub-modules, and multiple target access devices in the fiber optic access network run the sub-modules in the preset inference model respectively to execute the inference task. This eliminates the need to send the inference task to the cloud-side device through a long link, making full use of the computing resources of the optical access device while avoiding long link transmission between the end, edge, and cloud, shortening the execution time of completing the entire inference task, and improving inference efficiency.

[0107] Figure 4 is a flowchart illustrating a method for executing an inference task according to an exemplary embodiment. The method is executed by the aforementioned inference task execution system. Exemplarily, the method includes the following steps S401-S404.

[0108] S401: The first optical access device acquires the inference task to be executed.

[0109] In one possible implementation, the first optical access device is an OLT device. The OLT device acts as an intermediary in the fiber optic access network, connecting end-side devices (such as optical gateways) to target networks (such as local area networks or wide area networks). As a core device in the fiber optic access network, the OLT device possesses significant computing resources. Therefore, based on the operational status of the optical access devices in the connected fiber optic access network, it can dynamically allocate sub-modules in a pre-defined inference model and schedule multiple target access devices in the fiber optic access network to execute their respective sub-modules, ensuring the smooth execution of the inference task.

[0110] In one possible implementation, the inference task refers to the process of using a large model to perform analysis and processing on data, such as prediction, classification, or generation. This large model can be a large language model, a diffusion model, or something similar. For example, inference tasks can include intelligent question answering and image generation.

[0111] For example, intelligent question answering refers to intelligent applications that can understand users' natural language questions and provide accurate answers. Intelligent question answering utilizes natural language processing technology, knowledge graphs, machine learning algorithms, and more to analyze, understand, and reason about the questions posed by users, thereby returning the corresponding answers to the users. For example, users might query the weather or traffic conditions.

[0112] For example, image generation refers to the process of generating images using computer algorithms and models based on user-input data, such as text descriptions, image styles, etc. For instance, in text-based image generation applications, a user can input descriptive text, such as "green mountains and clear waters under blue skies and white clouds." The inference task execution system will then use an inference model (such as a deep learning model) to generate an image that matches the text description based on the content of this text.

[0113] In one possible implementation, the first optical access device can obtain the inference task to be executed by receiving an inference task request sent by the gateway device.

[0114] Among them, the reasoning task request is used to request the execution of a reasoning task.

[0115] The aforementioned gateway device can be any gateway device that accesses the target network (such as a local area network, wide area network, etc.) through the first optical access device. For example, the gateway device can be an ONU device, an ONT device, an FTTR gateway device, or other end-side device. Here, the FTTR gateway device mainly refers to the FTTR master gateway device; however, theoretically, it can also be an FTTR slave gateway device.

[0116] Optionally, the user can send an inference task request to the gateway device via user instructions, and the gateway device can forward the inference task request to the first optical access device.

[0117] For example, user commands include voice commands. A voice command refers to a user issuing an inference task request to the gateway device via voice. The gateway device includes a microphone for receiving user voice commands. The gateway device acquires the voice commands through the microphone and extracts the inference task request based on its voice recognition function. For example, the user's voice command could be "What's the weather like tomorrow?" or "Play music," etc.

[0118] For example, user instructions include text instructions. Text instructions refer to inference task requests issued by the user to the gateway device in written form. The gateway device includes input devices such as a keyboard and touchscreen to receive user text instructions. The gateway device obtains the text instructions through the input devices, parses the text, and extracts the inference task request.

[0119] For example, user instructions include gesture instructions. A gesture instruction refers to a user sending an inference task request to the device through a specific hand gesture. The gateway device includes a camera for capturing the user's gesture instructions. The gateway device captures the user's gesture instructions through the camera, parses the gesture instructions, and extracts the inference task request. For example, a user might instruct the playback of music by waving their hand in a preset gesture.

[0120] Optionally, users can also input the above user instructions on a terminal device (such as a mobile phone, computer, etc.), and the terminal device can send an inference task request to the first optical access device through the gateway device.

[0121] In another possible implementation, the first optical access device can also receive inference task requests from other devices in the fiber optic access network and obtain inference tasks to be executed. These other devices can be side-side devices (such as OLT devices), devices in the metropolitan area network (such as routers and switches), devices in the core network (such as servers), etc., excluding the first optical gateway device.

[0122] In another possible implementation, the user can also directly send an inference task request to the first optical access device, instructing the first optical access device to perform an inference task (such as a network diagnostic task). For example, the first optical access device includes input devices, such as a touchscreen display, microphone, keyboard, etc., and the user sends the inference task request to the first gateway device by operating the input devices within the first optical access device.

[0123] Of course, the inference task can also be a pre-set task in the first optical access device. For example, the first optical access device can periodically perform network diagnostics on the optical access network, or perform network diagnostics when there is a network fault, etc.

[0124] S402: The first optical access device sends task execution instructions to multiple target access devices.

[0125] The task execution instruction is used to instruct the target access devices to process the input information by running the corresponding target sub-module in the preset inference model. In this way, the first optical access device executes the inference task by instructing each target access device to run its corresponding target sub-module and obtain the execution result.

[0126] The target access device is the optical access device selected by the first optical access device from a plurality of second optical access devices.

[0127] In one possible implementation, S402 above sends the task execution instruction in the following manner:

[0128] a1: The first optical access device determines the time sequence of the target sub-modules to be run among the target access devices based on the connection relationship of the target sub-modules to be run among the target access devices in the preset inference model.

[0129] In this way, by clarifying the time sequence of the target sub-modules running among the target access devices, it can be ensured that the preset logic and process in the preset inference model can be correctly executed by each target access device, avoiding the failure of the inference task due to the disorder of the execution order of the target sub-modules, and improving the accuracy and reliability of the inference task execution.

[0130] Optionally, the connection relationships between target sub-modules in the preset inference model change as the internal structure of the preset inference model changes. The connection relationships between target sub-modules can be series, parallel, or mixed.

[0131] For example, concatenation refers to multiple sub-modules being connected sequentially in a predetermined order, with the output of one sub-module serving as the input of the next. For instance, in a text classification task in natural language processing, a pre-defined inference model first converts the text into a vector representation using a word embedding layer, and then processes it sequentially through sub-modules such as convolutional layers, pooling layers, and fully connected layers to obtain the final output classification result. In this case, the multiple sub-modules are connected in a concatenated manner.

[0132] Figure 5 is a schematic diagram of the connection relationship between target submodules in a preset inference model. Figure 5 includes two target submodules, namely target submodule 1 and target submodule 2. Target submodule 1 and target submodule 2 are connected in series. In this preset inference model, the output information 1 of target submodule 1 is input to target submodule 2, and target submodule 2 calculates the output information 2 from this output information 1. Therefore, the execution time sequence of target submodule 1 is earlier than that of target submodule 2.

[0133] For example, parallel processing can allow at least two sub-modules to simultaneously receive the same input and perform calculations independently. Parallel processing can also allow two sub-modules to have no correlation between their inputs and outputs, performing calculations completely independently. For instance, in multimodal learning, a pre-defined inference model may need to process various types of data, such as text, images, and audio. In this case, the pre-defined inference model can be divided into multiple sub-modules, each processing different types of data, with completely independent inputs and outputs between each sub-module.

[0134] Figure 6 illustrates another connection relationship between target submodules in a preset inference model. Figure 6 includes two target submodules: target submodule 1 and target submodule 2. Target submodule 1 and target submodule 2 are connected in parallel. In this preset inference model, target submodule 1 and target submodule 2 have the same input information, input information 1. Target submodule 1 and target submodule 2 perform calculations based on input information 1 to obtain their respective output information, output information 1 and output information 2. Therefore, the execution order of target submodule 1 can be the same as that of target submodule 2, meaning that target submodule 1 and target submodule 2 can be run simultaneously and independently.

[0135] For example, in addition to series and parallel connections, sub-modules in the preset inference model can also be connected in a more complex hybrid manner, that is, the connection relationship between sub-modules includes both series and parallel connections.

[0136] Figure 7 illustrates the connection relationship of another target submodule in a preset inference model. Figure 7 includes three target submodules: target submodule 1, target submodule 2, and target submodule 3. In this preset inference model, target submodule 1 and target submodule 2 are connected in parallel, while target submodule 1 and target submodule 2 are connected in series as a whole with target submodule 3. Target submodule 1 and target submodule 2 have the same input information, i.e., input information 1. Target submodule 1 and target submodule 2 perform calculations based on input information 1 to obtain their respective output information, i.e., output information 1 and output information 2. Target submodule 3 performs calculations based on the output information 1 and output information 2 of target submodule 1 and target submodule 2 to obtain the output information 3 of target submodule 3. Therefore, the execution order of target submodule 1 can be the same as that of target submodule 2, meaning that target submodule 1 and target submodule 2 can be run simultaneously and independently. Furthermore, the execution order of target submodule 1 and target submodule 2 is earlier than the execution order of target submodule 3.

[0137] How to break down the preset inference model into multiple sub-modules will be described in the second part of the embodiments of this application, and will not be elaborated here.

[0138] a2: The first optical access device sends task execution instructions to each target access device in chronological order.

[0139] The task execution instruction carries input information and is used to instruct the target access device to process the input information by running the corresponding target sub-module.

[0140] In this context, the input information of the submodules on the later target access devices is determined based on the output information of the submodules on the earlier target access devices. Continuing with Figure 7 as an example, the order in which target submodule 1 and target submodule 2 are run is earlier than the order in which target submodule 3 is run. The input information of target submodule 3 is the output information 1 of target submodule 1 and the output information 2 of target submodule 2.

[0141] In this way, the output of each submodule becomes the input of the subsequent submodules. Therefore, any data loss or error may affect the accuracy of the final execution result. By using the first optical access device as a scheduling device, the first optical access device is responsible for acquiring and managing the output information of each submodule. The first optical access device can orderly transmit the output information to the target optical access device corresponding to the subsequent submodule according to the execution order of the submodules in the preset inference model. This effectively reduces data loss or error during transmission, ensures the integrity and accuracy of the data, and avoids the impact of data problems on the final execution result, thereby enhancing the reliability of the inference task.

[0142] In one possible implementation, the second optical access device is an optical access device capable of communicating with the first optical access device.

[0143] Optionally, the second optical access device is an optical gateway device (such as an ONU device, ONT device, FTTR gateway device, etc.) that accesses the target network (such as a local area network, wide area network, etc.) through the first optical access device. For example, the first optical access device is an OLT device, and the second optical access device is an FTTR gateway device in the optical fiber access network. In this way, the computing resources of the optical access devices in the optical fiber access network are fully utilized, enabling the optical access devices to run some sub-modules in the preset inference model, thereby realizing distributed computing of the inference task.

[0144] In another possible implementation, the first optical access device can also be the target access device. That is, in addition to acting as a control and scheduling device to instruct other optical access devices to execute their respective sub-modules during the execution of the inference task, the first optical access device can also be used to execute sub-modules in the preset inference model.

[0145] How the first optical access device determines multiple target access devices (also called target access devices) among the optical access devices in the optical fiber access network will be described in the third part of the embodiments of this application later, and will not be repeated here.

[0146] S403: The target access device receives the task execution instruction and, in response to the task execution instruction, runs the target sub-module and obtains the output information of the target sub-module.

[0147] The output information is the calculation result of the target submodule on the input information.

[0148] S404: Return the output information to the first optical access device.

[0149] In one possible implementation, the first optical access device uses the output information of the last submodule in the preset inference model as the execution result of the inference task.

[0150] Figure 8 is a schematic diagram of a scenario where a first optical access device sends a task execution command and receives output information from each target access device. In Figure 8, the inference task is completed through three target access devices: target access device 1, target access device 2, and target access device 3. The connection relationship between the target sub-modules corresponding to each target access device in the preset inference model is serial, that is, the connection relationship between target sub-module 1, target sub-module 2, and target sub-module 3 is serial. The first optical access device first sends the inference task to the target access device 1 corresponding to the first sub-module (target sub-module 1) in the preset inference model. Then, the target access device 1 corresponding to the first sub-module (target sub-module 1) runs the first sub-module (target sub-module 1), obtains the output information 1 of the first sub-module, and returns the output information 1 to the first optical access device. Next, the first optical access device continues to send the output information 1 of the first sub-module to the target access device 2 corresponding to the second sub-module (target sub-module 2) in the preset inference model. The target access device 2 corresponding to the second sub-module (target sub-module 2) runs the second sub-module (target sub-module 2), obtains the output information 2 of the second sub-module (target sub-module 2), and returns it to the first optical access device. This process continues until the first optical access device receives the output information 3 of the target access device 3 corresponding to the last sub-module (target sub-module 3) in the preset inference model. Based on the output information 3 of the target access device 3 corresponding to the last sub-module (target sub-module 3), the first optical access device determines the execution result of the preset inference model and returns it to the gateway device.

[0151] Figure 9 illustrates a scenario where a first optical access device sends task execution instructions and receives output information from each target access device. In Figure 9, the inference task is completed through three target access devices: target access device 1, target access device 2, and target access device 3. The target submodule 1 corresponding to target access device 1 and the target submodule 2 corresponding to target access device 2 are connected in parallel within the preset inference model. The outputs of target submodule 1 and target submodule 2 serve as the inputs to target submodule 3 corresponding to the target access device. The first optical access device first sends the inference task to target access device 1 and target access device 2 corresponding to target submodule 1 in the preset inference model. Then, target access device 1 runs target submodule 1 and obtains output information 1. Target access device 2 runs target submodule 2 and obtains output information 2. Target access device 1 and target access device 2 return output information 1 and output information 2 to the first optical access device, respectively. The first optical access device then sends output information 1 and output information 2 to target access device 3 corresponding to target submodule 3 in the preset inference model. Target access device 3 runs target submodule 3 and obtains output information 3, which is returned to the first optical access device. The first optical access device determines the execution result of the preset inference model based on output information 3 and returns it to the gateway device.

[0152] In one possible implementation, when the first optical access device receives an inference task request from the gateway device, the first optical access device returns the execution result of the inference task to the gateway device after receiving the execution result, and the gateway device then feeds back the execution result to the user.

[0153] The above describes a method where the first optical access device acts as an intermediary for transmitting the inputs and outputs of each target access device. The first optical access device instructs multiple target access devices to sequentially execute their respective target sub-modules, thereby obtaining the execution result of the inference task. Furthermore, in a fiber optic access network where optical access devices can communicate with each other, the first optical access device can also send the inference task to the target access device corresponding to the first sub-module in the preset inference model. After executing its corresponding target sub-module, the target access device corresponding to the first sub-module directly transmits its output information to the target access device corresponding to the second sub-module in the preset inference model. After receiving the output information, the target access device corresponding to the last sub-module in the preset inference model returns the output information to the first optical access device. This eliminates the need for data transmission between the target access devices and the first optical access device, further shortening the execution time of the inference task.

[0154] In the first part of this application's embodiment, the first optical access device breaks down a complex preset inference model into multiple sub-modules and assigns them to target access devices in the optical fiber access network for processing. Each target access device is only responsible for a portion of the computation in the preset inference model. Firstly, this eliminates the need to send the inference task to the cloud-side device via a long link, fully utilizing the computing resources of the optical access device while avoiding long-distance end-edge-cloud transmission, shortening the execution time of the entire inference task and improving inference efficiency. Secondly, different optical access devices may have different computing resources. By rationally allocating the sub-modules in the preset inference model, the computing resources in the optical fiber access network can be utilized more effectively, helping to achieve load balancing and preventing overload or idleness of optical access devices. Thirdly, since the preset inference model is broken down into multiple sub-modules running on different optical access devices, even if one optical access device fails, the entire inference task will not fail. Other optical access devices can continue to execute the sub-modules corresponding to the failed device, enabling flexible response to failure crises and improving the reliability of the inference task.

[0155] The above is the first part of the embodiments of this application. The following, with reference to Figures 10 and 11, describes how to divide the preset inference model into multiple sub-modules.

[0156] In some embodiments, the preset inference model is split into multiple sub-modules according to its internal structure. A target access device in the fiber optic access network is used to execute the corresponding sub-module, so that the optical access device in the fiber optic access network can still use the preset inference model for model inference even when computing resources are limited, while making full use of the computing resources of the optical access device in the fiber optic access network.

[0157] In some examples, the pre-defined inference model includes multiple first-level submodules connected sequentially; each first-level submodule includes one or more second-level submodules, and these second-level submodules are connected in parallel. Each target access device (also referred to as the target access device) operates a target submodule including at least one first-level submodule and / or at least one second-level submodule.

[0158] Figure 10 is a schematic diagram of the structure of a preset inference model containing multiple sub-modules. In Figure 10, the preset inference model includes n first-level sub-modules (i.e., first-level sub-module 1, first-level sub-module 2... first-level sub-module n) connected in sequence. Each first-level sub-module contains m second-level sub-modules (i.e., second-level sub-module 1, second-level sub-module 2... second-level sub-module m), and the m second-level sub-modules are connected in parallel.

[0159] Continuing with Figure 7 as an example, target submodule 1 and target submodule 2 constitute first-level submodule a, and target submodule 3 constitutes first-level submodule b. First-level submodule a contains two second-level submodules: target submodule 1 and target submodule 2. First-level submodule b contains one second-level submodule: target submodule 3.

[0160] Optionally, the multiple target access devices include a first device, a second device, and a third device. The first and second devices are used to run their respective target sub-modules in parallel. The first and third devices are used to run their respective target sub-modules serially. In this way, by running the target sub-modules on the first and second devices in parallel, the processing capabilities of multiple optical access devices can be fully utilized, processing different parts of the inference task simultaneously, significantly reducing the total execution time of the inference task and improving processing speed.

[0161] For example, the first device, the second device, and the third device are used to run different secondary sub-modules; and the secondary sub-modules run by the first device and the second device belong to the same primary sub-module, while the secondary sub-modules run by the first device and the third device belong to different primary sub-modules.

[0162] Figure 11 illustrates the relationship between the first, second, and third devices. In Figure 11, device 1 runs target submodule 1 from Figure 7, device 2 runs target submodule 2, and device 3 runs target submodule 3. Target submodules 1 and 2, as second-level submodules, are connected in parallel within the preset inference model. Target submodule 3, as a first-level submodule (b), is connected in series with first-level submodule a, where target submodules 1 and 2 reside. That is, device 1 and device 2 act as the first and second devices, respectively, and can run their respective target submodules in parallel. When the device group consisting of devices 1 and 2 and device 3 need to run their respective target submodules serially, the device group consisting of devices 1 and 2 acts as the first device, and device 3 acts as the third device.

[0163] For example, the input information of the target submodule on the first device and the second device is the same, and the input of the target submodule on the third device is determined based on the output of the target submodule on the first device and the second device. For instance, the input information of the target submodule on the first device and the second device comes from the output information obtained from the previous target submodule.

[0164] Of course, the input information of the target submodules on the first and second devices can also be different. For example, when text and images need to be processed separately in the preset inference model, the input of the first device is text, and the input of the second device is an image. The target submodule in the third device needs to perform further calculations based on the calculation results of the text obtained by the target submodule in the first device (such as the text embedding vector) and the calculation results of the image obtained by the target submodule in the second device (such as the feature map of the image). In this case, the input information of the first device is not the same as the input information of the second device, and the target submodule in the first device can be computed in parallel with the target submodule in the second device.

[0165] In the second part of this application's embodiments, the preset inference model is divided into multiple first-level sub-modules. Within each first-level sub-module, parallel connections between multiple second-level sub-modules allow them to process data simultaneously, significantly improving the parallel processing capability for performing inference tasks. Especially when processing large-scale datasets or complex inference tasks, parallel processing can significantly reduce overall processing time and improve the response speed and throughput of the inference task. Furthermore, by dividing the inference model into multiple first-level sub-modules, each of which can contain multiple parallel second-level sub-modules, this modular design offers high flexibility and scalability. When it is necessary to adjust the complexity of the inference task or add new functionality, first-level or second-level sub-modules can be easily added, deleted, or replaced without requiring major modifications to the entire preset inference model. In addition, the modular design enhances fault isolation and tolerance performance. If a first-level or second-level sub-module fails, the problem can be quickly located and the faulty module isolated without affecting the normal operation of other sub-modules.

[0166] The above is the second part of the embodiments of this application. The following, with reference to FIG12, describes the specific process by which the first optical access device determines multiple target access devices in the optical fiber access network.

[0167] In some embodiments, the first optical access device determines the target access device and the sub-modules that each target access device needs to run based on the model running capability of the optical access devices in the optical fiber access network. This ensures that the sub-modules allocated to the target access device are matched with the target access device, thus ensuring that the target access device can execute the sub-modules smoothly and avoiding the waste of the target access device's computing resources.

[0168] Figure 12 is a flowchart illustrating a first optical access device determining a plurality of target access devices according to an exemplary embodiment. Exemplarily, the method includes the following steps S1201-S1202.

[0169] S1201: The first optical access device acquires the target operation information of the optical access device.

[0170] Among them, the target operation information is used to indicate the model operation capability of the optical access equipment.

[0171] For example, target operational information includes memory data, which characterizes the memory usage in the optical access device. Memory data includes, but is not limited to, memory size, memory utilization, etc.

[0172] For example, the target operational information includes computing power data, which characterizes the computing power of the optical access device. Specifically, the computing power data could be the number of floating-point operations per second supported by the optical access device.

[0173] For example, the target operational information includes CPU data, which characterizes the CPU usage in the optical access device. For instance, CPU data could include CPU utilization, CPU temperature, and so on.

[0174] Understanding the memory, computing power, and CPU usage of optical access devices helps the primary optical access device allocate inference tasks more rationally. For example, tasks requiring significant computing resources can be prioritized for optical access devices with higher computing power; tasks requiring fast response times can be prioritized for optical access devices with lower CPU utilization; and tasks requiring substantial storage space can be prioritized for optical access devices with larger memory. This optimized task allocation strategy helps improve the efficiency and accuracy of task execution.

[0175] S1202: Based on the model operation capabilities of the optical access equipment, determine multiple target access devices and the target sub-modules that each target access device needs to run.

[0176] Among them, the model operation capability of the target access device is consistent with the capability requirements of the target sub-module corresponding to the target access device.

[0177] For example, when the target operating information includes memory data, the first optical access device matches the memory requirements of the submodules with the memory sizes of each optical access device to ensure that the optical access devices can provide sufficient memory resources to support the operation of the submodules. For instance, if the target access device has a large amount of memory, a submodule requiring more memory can be allocated to that target access device. If the target access device has a small amount of memory, a submodule requiring less memory can be allocated to that target access device.

[0178] For example, when the target operating information includes computing power data, the first optical access device matches the computing power of each optical access device with the computing power requirements of the sub-module to ensure that the optical access device can execute the sub-module efficiently.

[0179] For example, when the target operating information includes CPU data, the first optical access device matches the CPU data of each optical access device with the CPU requirements of the submodule to ensure that the CPU of the optical access device can meet the performance requirements of executing the submodule. If the CPU utilization of an optical access device is too high or the CPU temperature is too high, the first optical access device will not allocate the submodule to that optical access device to reduce the load on that optical access device and avoid CPU performance degradation or hardware damage.

[0180] In other embodiments, the first optical access device is further configured to receive configuration information. This configuration information includes device identifiers of multiple target access devices and the number of submodules in a preset inference model. The device identifiers indicate the optical access devices in the fiber optic access network. The number of submodules indicates the total number of submodules in the preset inference model.

[0181] After receiving the configuration information, the first optical access device determines the target access devices and the target sub-modules corresponding to each target access device in the optical fiber access network based on the configuration information.

[0182] In this way, users can determine the configuration information according to different application scenarios and needs, and send the configuration information to the first optical access device. Based on the configuration information, the first optical access device can quickly and flexibly configure the target access device and the corresponding sub-modules, shorten the deployment time of the first optical access device for the target access device and the corresponding sub-modules, simplify the deployment process of the first optical access device, and achieve rapid deployment to complete the inference task.

[0183] In the third part of this application's embodiments, by acquiring the target operating information of the optical access devices, the first optical access device can accurately understand the model operating capabilities of each optical access device. This enables the first optical access device to allocate sub-modules according to actual capability requirements, ensuring that each target access device can run sub-modules that match its capabilities, thereby avoiding wasted computing resources or overload, and ensuring that inference tasks can be executed sequentially.

[0184] The above constitutes the third part of the embodiments of this application. The following, with reference to Figures 13 and 14, describes the specific implementation method of communication between the first optical access device and each of the second optical access devices.

[0185] In some embodiments, the first optical access device communicates with optical access devices in the optical fiber access network via a preset communication protocol. Through this preset communication protocol, the first optical access device schedules the optical access devices in the optical fiber access network, enabling each device to execute its corresponding sub-module, thereby achieving model inference.

[0186] In one possible implementation, the first optical access device and the second optical access device communicate via the OMCI protocol.

[0187] An OMCI protocol message consists of the following parts: a generic encapsulation method (GEM) header, a transaction-related identifier, a message type, a device identifier, a message identifier, a message content, and an OMCI tail field. Figure 13 is a schematic diagram of an OMCI protocol message format. In Figure 13, the GEM header occupies 5 bytes, the transaction-related identifier occupies 2 bytes, the message type occupies 1 byte, the device identifier occupies 1 byte, the message identifier occupies 4 bytes, the message content occupies 32 bytes, and the OMCI tail field occupies 8 bytes.

[0188] The GEM header identifies the beginning of the message. The Transaction Related Identifier ensures that a set of corresponding request and response messages match correctly; that is, it ensures that the task execution instruction sent by the first optical access device matches the output information returned by the second optical access device. The value of this field must be consistent across a set of request and response messages. The Message Type indicates the specific type of the OMCI message, such as obtaining target operating information for the optical access device or instructing the second optical access device to load the corresponding target submodule. The Device Identifier indicates which optical access device is being used. The Message Identifier uniquely identifies each OMCI message, ensuring correct processing and response. The Message Content contains the actual content of the message, such as the target operating information of the optical access device and the submodules that the optical access device needs to run. The OMCI tail field typically contains cyclic redundancy check (CRC) information to ensure the integrity and correctness of the message.

[0189] Figure 14 is a schematic diagram of the field composition of a message type. In Figure 14, the message type includes: destination bit (DB), action request (AR), acknowledgement (AK), and message type (MT). The destination bit is usually fixed at 0 and is not used to indicate a specific operation or purpose in OMCI messages; instead, it is reserved as part of the protocol. The action request (AR) is a flag indicating whether the first optical access device needs a response from other optical access devices in the fiber optic access network regarding the execution result of the requested action (such as the output information after the optical access device runs the target submodule). The acknowledgement is also a flag indicating whether the message is an instruction sent by the first optical access device to another optical access device, or information returned by one optical access device to the first optical access device. The message type indicates the scheduling content of the first optical access device for other optical access devices in the fiber optic access network (such as obtaining target operation information of optical access devices, instructing a second optical access device to load the corresponding target submodule, etc.).

[0190] For example, when the message type of the OMCI message sent by the first optical access device is 8, the OMCI message can be used by the first optical access device to send a task execution instruction to the second optical access device, thereby instructing the second optical access device to process the input information by running the corresponding target submodule.

[0191] For example, when the message type of the OMCI message sent by the first optical access device is 9, the OMCI message can be used by the first optical access device to obtain the target operation information of the optical access device, thereby confirming whether the optical access device is idle and whether it can provide computing resources to run the submodule.

[0192] For example, when the message type of the OMCI message is 19, the OMCI message is used by the second optical access device to send the output information obtained from running the target submodule to the first optical access device, or by the first optical access device to send the inference task to the first second optical access device, or by the first optical access device to send the output information obtained from the previous second optical access device to the next second optical access device.

[0193] For example, when the message type of the OMCI message is 20, the OMCI message is used to transmit partial data, such as partial output information. For example, when the message type of the OMCI message is 21, the OMCI message is used to indicate the end of data transmission.

[0194] In the fourth part of the embodiments of this application, the communication between the first optical access device and the devices in the optical fiber access network is realized through the OMCI protocol, so that the second optical access device can execute its corresponding target sub-module under the scheduling of the first optical access device, thereby realizing model inference in the devices of the optical fiber access network.

[0195] This application also provides an execution system for inference tasks. The system includes a first optical access device and multiple second optical access devices; the second optical access devices access a target network through the first optical access device.

[0196] The first optical access device is used to acquire the inference task to be executed; and to send task execution instructions to multiple second access devices. The task execution instructions are used to instruct the second access devices to process the input information by running the corresponding target sub-module in the preset inference model.

[0197] The second optical access device is used to receive task execution instructions, and in response to the task execution instructions, run the target submodule, obtain the output information of the target submodule, and return the output information to the first optical access device.

[0198] This application also provides a computing device 100. As shown in FIG15, the computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other via the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 100.

[0199] Bus 102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 15, but this does not imply that there is only one bus or one type of bus. Bus 102 can include pathways for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).

[0200] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0201] Memory 106 may include volatile memory, such as random access memory (RAM). Processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0202] The memory 106 stores executable program code, and the processor 104 executes the executable program code to implement the functions of the execution system for the aforementioned reasoning task, thereby realizing the execution method of the reasoning task. That is, the memory 106 stores instructions for executing the execution method of the reasoning task.

[0203] The communication interface 108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 100 and other devices or communication networks.

[0204] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform an execution method for an inference task, or instruct the computing device to perform an execution method for an inference task.

[0205] This application also provides a chip. The chip integrates a control circuit for implementing the functions of the aforementioned computing device and one or more ports. Optionally, the functions supported by the chip can be referred to above, and will not be repeated here. Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, random access memory, etc. The aforementioned processing unit or processor can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0206] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform an execution method for a reasoning task.

[0207] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of this application, such as but not limited to the memory, computer-readable storage medium and communication chip, are all non-transitory.

[0208] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0209] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0210] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method of performing an inference task, the method comprising: The method is performed by a first optical access device in the optical fiber access network, and the method includes: Obtain the reasoning task to be executed; Obtain the execution result of the inference task; the execution result is obtained by running different sub-modules in the preset inference model through multiple target access devices to execute the inference task; The target access device includes other optical access devices capable of communicating with the first optical access device.

2. The execution method of claim 1, wherein, The process of obtaining the reasoning task to be executed includes: The system receives an inference task request sent by a gateway device; the gateway device is any gateway device that accesses the target network through the first optical access device; the inference task request is used to request the execution of the inference task. The method further includes: The execution result of the inference task is returned to the gateway device.

3. The execution method according to claim 1 or 2, characterized in that, The target access device is an optical gateway device that accesses the target network through the first optical access device.

4. The execution method according to any one of claims 1 to 3, characterized in that, The first optical access device is an optical line terminal (OLT).

5. The execution method according to any one of claims 1-4, characterized in that, Obtaining the execution result of the inference task includes: From the optical access devices in the optical fiber access network, determine the plurality of target access devices and the target sub-modules that each target access device needs to run; The inference task is executed by instructing each of the target access devices to run its corresponding target submodule, and the execution result is obtained.

6. The execution method of claim 5, wherein, From the optical access devices in the optical fiber access network, determine the plurality of target access devices and the target sub-modules that each target access device needs to run, including: Obtain the target operation information of the optical access device, wherein the target operation information is used to indicate the model operation capability of the optical access device; Based on the model operation capabilities of the optical access device, the plurality of target access devices and the target sub-modules that each target access device needs to run are determined, wherein the model operation capabilities of the target access devices are consistent with the capability requirements of the target sub-modules corresponding to the target access devices.

7. The execution method according to claim 5 or 6, characterized in that, The step of instructing each of the target access devices to run its corresponding target sub-module to execute the inference task and obtain the execution result includes: Based on the connection relationship of the target sub-modules that each target access device needs to run in the preset inference model, determine the time sequence of the target sub-modules running among the target access devices; In accordance with the time sequence, task execution instructions are sent sequentially to each of the target access devices. The task execution instructions carry input information and are used to instruct the target access devices to process the input information by running the corresponding target sub-modules. The input information of the sub-modules on the later target access devices is determined based on the output information of the sub-modules on the earlier target access devices.

8. The execution method of claim 7, wherein, The step of instructing each of the target access devices to run its corresponding target sub-module to execute the inference task and obtain the execution result further includes: Obtain the output information of the submodule returned by the target access device; The output information of the last submodule in the preset reasoning model is used as the execution result of the reasoning task.

9. The execution method according to any one of claims 6-8, characterized in that, The target operating information includes at least one of memory data, computing power data, and central processing unit (CPU) data; the memory data is used to characterize the memory usage in the optical access device, the computing power data is used to characterize the computing power of the optical access device, and the CPU data is used to characterize the CPU usage in the optical access device.

10. The execution method according to any one of claims 1-9, wherein, The preset inference model includes multiple first-level sub-modules connected in sequence; each first-level sub-module includes one or more second-level sub-modules, and the multiple second-level sub-modules are connected in parallel; the target sub-module operated by each target access device includes at least one of the first-level sub-modules and / or at least one of the second-level sub-modules.

11. The execution method according to any one of claims 1-10, wherein, The plurality of target access devices include a first device, a second device, and a third device; wherein, the first device and the second device are used to run their respective target sub-modules in parallel; and the first device and the third device are used to run their respective target sub-modules in serial order.

12. The execution method of claim 11, wherein, The input information of the target submodule on the first device and the second device is the same; the input of the target submodule on the third device is determined based on the output of the target submodule on the first device and the second device.

13. The execution method according to claim 11 or 12, characterized in that, The first device, the second device, and the third device are each used to run different secondary sub-modules; and the secondary sub-modules run by the first device and the second device belong to the same primary sub-module, while the secondary sub-modules run by the first device and the third device belong to different primary sub-modules.

14. The execution method according to any one of claims 1 to 13, wherein, The method further includes: Receive configuration information; the configuration information includes the device identifiers of the plurality of target access devices and the number of sub-modules in the preset inference model; the device identifiers are used to indicate the optical access devices in the optical fiber access network; the number of sub-modules is used to indicate the total number of sub-modules in the preset inference model.

15. The execution method according to any one of claims 1 to 14, wherein, The target access device also includes a first optical access device.

16. An execution method of an inference task, the method comprising: The method is performed by a second optical access device in the optical fiber access network, and the method includes: Obtain task execution instructions; the task execution instructions are used to instruct the input information to be processed by running the target sub-module in the preset inference model; In response to the task execution instruction, the target submodule is run to obtain the output information of the target submodule; the output information is the calculation result of the target submodule on the input information.

17. The execution method of claim 16, wherein, The process of obtaining the task execution instruction includes: Receive the task execution instruction sent by the first optical access device; The method further includes: The output information of the target submodule is sent to the first optical access device; the second optical access device accesses the target network through the first optical access device.

18. An inference task execution system, comprising: The system includes a first optical access device and multiple second optical access devices; the second optical access devices access the target network through the first optical access device. The first optical access device is used to acquire the inference task to be executed; A task execution instruction is sent to the plurality of second access devices, the task execution instruction being used to instruct the second access devices to process the input information by running the corresponding target sub-module in the preset inference model; The second optical access device is configured to receive the task execution instruction, and in response to the task execution instruction, run the target sub-module, obtain the output information of the target sub-module, and return the output information to the first optical access device.

19. A computing device, comprising: include: Processor and memory; The processor is connected to a memory for storing computer execution instructions, and the processor executes the computer execution instructions stored in the memory to enable the computing device to implement the method as described in any one of claims 1-17.

20. A computer-readable storage medium, characterized in that, The device stores computer instructions that, when executed on a computing device, cause the computing device to perform the method as described in any one of claims 1 to 17.

21. A computer program product, characterised in that, include: A computer executes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-17.

Images