System and method for distributed artificial intelligence model training and / or inference

Abstract

The present disclosure relates to a system and method for distributed artificial intelligence (AI) model training and / or inference. Provided is a system for distributed AI model training and / or inference. The system comprises: a master node, which comprises an image sensor and an image signal processor; and at least one slave node coupled to the master node, each slave node comprising an image sensor, an image signal processor and a projector, wherein the slave node is configured to transmit to other slave nodes and / or to the master node, by means of projecting an image, information related to an execution result of at least one AI computing process that is assigned to the slave node.

Description

Systems and methods for training and / or inference of distributed artificial intelligence models Technical Field

[0001] This disclosure generally relates to distributed artificial intelligence (AI) systems, and more specifically, to optical device-based distributed AI model training and / or inference platforms. Background Technology

[0002] In recent years, artificial intelligence technologies, represented by deep learning and large-scale modeling, have been widely applied across various industries, greatly promoting the popularization of intelligent applications. However, with the continuous expansion of model size and the sustained growth of computational demands, the bottleneck of computing power has become increasingly prominent, especially in training and inference scenarios using private data. Existing technical solutions still have many shortcomings, specifically as follows:

[0003] (1) High equipment costs: The training and inference of current models heavily rely on computing devices such as high-performance graphics processing units (GPUs). Hardware investments ranging from tens of thousands of yuan per card to hundreds of millions of yuan per cluster mean that the research and deployment of artificial intelligence technology can only be dominated by computing power manufacturers or large Internet companies. The high cost of computing power not only limits the participation of small and medium-sized enterprises and research institutions, but also hinders the popularization of artificial intelligence technology, making this field a high-investment, high-risk industry.

[0004] (2) Insufficient scalability: Traditional computing devices are based on heterogeneous computing architectures such as CUDA and are heavily dependent on specific GPU hardware resources. Due to the architectural differences between different generations of GPUs, users are often forced to replace old devices to adapt to new requirements when faced with new large language models, resulting in resource waste and increased technology iteration costs.

[0005] (3) Conflict between storage and computing resources: Due to the physical characteristics of semiconductor devices (such as size, power consumption and cost), training and inference tasks usually cannot be executed in parallel. During online training, the inference service needs to be paused (similar to "half-duplex communication" mode), which leads to reduced device utilization and makes it difficult to meet the application scenarios with high real-time requirements.

[0006] (4) Low computational efficiency: The special characteristics of artificial intelligence network structure (such as sparse computing, dynamic branching, etc.) cause the utilization rate of computing resources of hardware devices to decrease significantly during forward propagation (inference) and backward propagation. A large amount of computing power is idle or wasted, which further exacerbates the contradiction between the supply and demand of computing power.

[0007] (5) Bottleneck in distributed training communication: In distributed artificial intelligence systems, the synchronization of data and model weights depends on high-speed network communication. However, hardware network latency, bandwidth limitations, and heterogeneity of node computing capabilities can significantly drag down training and inference efficiency, and even lead to performance degradation.

[0008] In summary, existing computing power solutions have significant shortcomings in terms of cost, scalability, resource scheduling, computational efficiency, and distributed collaboration. Therefore, a new technical solution is urgently needed to overcome the limitations of computing resources and achieve efficient, low-cost, and scalable artificial intelligence training and inference. Summary of the Invention

[0009] This section provides a brief overview of the present disclosure to offer a basic understanding of some aspects thereof. However, it should be understood that this overview is not an exhaustive summary of the present disclosure. It is not intended to identify key or essential parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is merely to present certain concepts of the disclosure in a simplified form as a prelude to the more detailed description that follows.

[0010] According to one aspect of this disclosure, a system for training and / or inferring a distributed artificial intelligence (AI) model is provided, comprising: a master node including an image sensor and an image signal processor; and at least one child node coupled to the master node, each child node including an image sensor, an image signal processor, and a projector, wherein the child node is configured to transmit information relating to the execution results of at least one AI computational process assigned to the child node to other child nodes and / or to the master node by projecting images.

[0011] According to another aspect of this disclosure, a method is provided performed by a system for training and / or inferring a distributed AI model, the system comprising a master node and at least one child node coupled to the master node, the master node comprising an image sensor and an image signal processor, and each child node comprising an image sensor, an image signal processor and a projector, wherein the method comprises: the child node transmitting information relating to the execution result of at least one AI computational processing assigned to the child node to other child nodes and / or to the master node by projecting an image.

[0012] According to another aspect of this disclosure, a method performed by a node device is provided, comprising: performing at least one AI computational process; and projecting an image via a projector, the projected image being used to convey information relating to the execution result of the at least one AI computational process.

[0013] According to another aspect of this disclosure, a node device is provided, including an image sensor, an image signal processor, and a projector, wherein the node device is configured to perform the method described above.

[0014] According to another aspect of this disclosure, a method performed by a node device is provided, comprising: controlling at least one child node device coupled to the node device to perform at least one AI task, each AI task including at least one corresponding AI computational process; and detecting a projected image of each child node device via an image sensor to receive information relating to the execution result of at least one AI computational process assigned to the child node device.

[0015] According to another aspect of this disclosure, a node device is provided, including an image sensor and an image signal processor, wherein the node device is configured to perform the method described above.

[0016] According to another aspect of this disclosure, a non-transient computer-readable storage medium is provided that stores computer-readable program instructions, which, when executed by one or more processors, cause any of the methods described above to be performed.

[0017] According to another aspect of this disclosure, a computer program product is provided, comprising computer-readable program instructions that, when executed by one or more processors, cause any of the methods described above to be performed. Attached Figure Description

[0018] This disclosure can be better understood by referring to the detailed description given below in conjunction with the accompanying drawings, in which the same or similar reference numerals are used throughout the drawings to denote the same or similar elements. All the drawings, together with the following detailed description, are incorporated in and form a part of this specification, and are used to further illustrate embodiments of this disclosure and explain the principles and advantages of this disclosure. Wherein:

[0019] Figure 1A shows a system architecture diagram of a system for distributed AI model training and / or inference according to an embodiment of the present disclosure.

[0020] Figure 1B shows an exemplary projected image according to an embodiment of the present disclosure.

[0021] Figure 2 shows an exemplary overall flowchart of a method for a system for training and / or inferring a distributed AI model according to embodiments of the present disclosure.

[0022] Figure 3 shows a flowchart of the method performed by the master node according to an embodiment of the present disclosure.

[0023] Figure 4 shows a flowchart of a method performed by a child node according to an embodiment of the present disclosure.

[0024] Figure 5 shows an example block diagram of an exemplary node device according to an embodiment of the present disclosure.

[0025] Figure 6 shows an example flowchart of a method performed by a child node device according to an embodiment of the present disclosure.

[0026] Figure 7 shows an example block diagram of an exemplary node device according to an embodiment of the present disclosure.

[0027] Figure 8 shows an example flowchart of a method performed by a node device according to an embodiment of the present disclosure.

[0028] Figure 9 shows a flowchart of a method for a system for distributed AI model training and / or inference according to an embodiment of the present disclosure.

[0029] Figure 10 shows a block diagram illustrating an exemplary hardware configuration of a computer system capable of implementing embodiments of the present invention.

[0030] The features and aspects of this disclosure will become clear from the following detailed description taken in conjunction with the accompanying drawings. Detailed Implementation

[0031] Various exemplary embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. For clarity and brevity, not all implementations of the embodiments are described in this specification. However, it should be noted that many implementation-specific settings can be made when implementing embodiments of this disclosure to meet specific needs and achieve the developer's specific goals. Furthermore, it should be understood that while development work may be complex and time-consuming, such development is merely a routine task for those skilled in the art who benefit from this disclosure.

[0032] Furthermore, it should be noted that, in order to avoid obscuring this disclosure with unnecessary details, only processing steps and / or device structures closely related to the technical solutions of this disclosure are shown in the accompanying drawings. The following description of exemplary embodiments is merely illustrative and is not intended to limit this disclosure or its application in any way.

[0033] To address the shortcomings of traditional training and / or inference platforms, this disclosure proposes a distributed artificial intelligence training and / or inference platform based on optical devices. The distributed artificial intelligence training and / or inference platform according to this disclosure includes a master node (e.g., including an image signal processor and an image sensor) and several child nodes (e.g., including an image signal processor, an image sensor, and a projector). The child nodes transmit information related to the results of AI computations performed at that child node to other child nodes and / or the master node by projecting images.

[0034] The distributed artificial intelligence training and / or inference platform disclosed herein uses the aforementioned nodes based on optical devices (such as image sensors and / or projectors) to form a network, which can effectively reduce the cost of AI model training and inference.

[0035] This disclosed distributed AI training and / or inference platform supports dynamic networking based on demand, exhibiting strong scalability. For example, when model updates require greater computing power, only new child nodes need to be added as needed, without replacing all the original equipment used for inference / training. Furthermore, when supporting inference / training of multiple AI models is required, flexible networking can be implemented for multiple AI models, enabling the simultaneous execution of different AI tasks by multiple AI models.

[0036] The distributed artificial intelligence training and / or inference platform disclosed herein can support information transmission between nodes via projected images, avoiding the limitations of traditional communication conditions.

[0037] The distributed artificial intelligence training and / or inference platform disclosed herein enables intermediate results (projected images) to be stored in physical space, thus eliminating storage resource limitations for intermediate execution results.

[0038] The distributed artificial intelligence training and / or inference platform disclosed herein can support simultaneous forward propagation computation (e.g., inference) and backward propagation computation (e.g., training and online learning), which can effectively improve computational efficiency.

[0039] The distributed artificial intelligence training and / or inference platform disclosed herein can support arbitrary adjustment of the network structure, thereby increasing parallelism and improving computational efficiency.

[0040] Figure 1A shows a system architecture diagram of a system 100 for training and / or inference of a distributed AI model according to an embodiment of the present disclosure.

[0041] This system 100, for example, implements a distributed artificial intelligence model training and / or inference platform based on optical devices (e.g., a projector). For illustration, Figure 1A shows a specific AI application scenario where the AI ​​model receives an image (e.g., the panda image shown in Figure 1A) as input and outputs a description of the recognized image content in natural language (e.g., the natural language "The image shows a panda sitting on a branch" shown in Figure 1A). This AI model is implemented, for example, using a Convolutional Neural Network (CNN) and a Large Language Model (LLM).

[0042] This application scenario can involve various AI tasks, such as AI model inference, AI model training, and AI model online learning. Each AI task can involve multiple AI computations, which can be executed by multiple sub-nodes in the system 100.

[0043] Although Figure 1A only illustrates one application scenario, it can be understood that system 100 can serve multiple application scenarios simultaneously; in other words, it can serve AI tasks related to various AI models.

[0044] As shown in Figure 1A, system 100 includes server 101, main node 102, and multiple child nodes 103-1 to 103-n. System 100 also includes multiple backup nodes 104-1 to 104-m. n and m are natural numbers greater than 1.

[0045] Although Figure 1A only shows the sub-nodes 103-1 to 103-n serving this application scenario, it can be understood that the system may include more sub-nodes (e.g., adding more sub-nodes when computational demands increase, or adding more sub-nodes when serving more application scenarios) or fewer sub-nodes (e.g., reducing the number of participating sub-nodes when computational demands decrease, or reducing the number of participating sub-nodes when the number of application scenarios decreases).

[0046] Server 101 can group child nodes for AI tasks related to specific AI models, thereby constructing one or more pipelined and / or parallelized AI computing links. Taking the application scenario in Figure 1A as an example, server 101 can network child nodes 103-1 to 103-n based on the network structure of the AI ​​model to be run, and send the network information to the master node 102, and / or send at least a portion of the network information to each child node 103-1 to 103-n.

[0047] Network information can include the topology between node devices and AI computing task allocation information.

[0048] A single node device can serve one or more AI tasks, for example, performing multiple AI computations corresponding to different AI tasks. This is particularly advantageous when the system 100 simultaneously serves different application scenarios. The multiple AI tasks processed in parallel can be associated with different AI models serving different application scenarios.

[0049] The same node device can also serve the computation of different propagation directions of an AI task. For example, it can be used to perform forward propagation computation and backward propagation computation of the same AI task.

[0050] It is understood that AI computing links involving multiple sub-nodes can have various network structures, not limited to the linear chain shown in Figure 1A. Alternatively, AI computing links can include tree structures or mesh structures with multiple levels. Advantageously, multiple sub-nodes in the AI ​​computing link can be arranged according to the network structure or hierarchical structure of the AI ​​model, so that the AI ​​computing link can better perform parallel computing.

[0051] Server 101 can distribute the network structure and / or weight parameters of the AI ​​model to the relevant child nodes. For example, as shown in Figure 1A, server 101 can distribute parameters related to layer blocks 1-n to the corresponding child nodes 103-1 to 103-n, and each child node uses the parameters it receives to perform the AI ​​computation processing assigned to it.

[0052] The master node 102 may include an image signal processor and an image sensor. The master node 102 is coupled to the various child nodes and to the server 101. In the example of Figure 1A, the master node 102 controls the child nodes 103-1 to 103-n to perform AI tasks.

[0053] Each child node and backup node may include an image signal processor, an image sensor, and a projector. Each child node can transmit information to other child nodes and / or the main node by projecting an image. As shown in Figure 1A, child node 103-1 transmits information to other child nodes and / or the main node by projecting an image 105. That is, other child nodes and / or the main node can use the image sensor to detect the image 105 projected by child node 103-1 to obtain information. The projected image 105 may include information related to intermediate execution results obtained by performing AI computation processing assigned to child node 103-1.

[0054] In embodiments according to this disclosure, the relationships between nodes can be represented as upstream and / or downstream. For example, for child node 103-1 in FIG. 1A that projects the projected image 105, child node 103-2 that receives the projected image 105 to obtain information is its downstream child node. Conversely, for child node 103-2, child node 103-1 is its upstream child node. It should be noted that, depending on the needs, a child node may have multiple downstream child nodes and multiple upstream child nodes. Furthermore, the concepts of upstream and downstream can be determined based on the AI ​​computational processing performed by the node. For example, for one AI computational processing being performed (e.g., AI computational processing related to forward propagation computation), child node 103-1 may be the upstream node of child node 103-2; at the same time, for another AI computational processing being performed in parallel (e.g., AI computational processing related to backpropagation computation), child node 103-1 may be the downstream node of child node 103-2.

[0055] The master node 102 can monitor the status of child nodes via a wireless network, and can also monitor the status of child nodes by detecting projected images of child nodes using an image sensor. When the master node 102 detects that a child node needs to be replaced (e.g., when a child node is abnormal or when some child nodes need to be replaced in response to receiving an online learning request), it can activate one or more of the backup nodes 104-1 to 104-m to replace the child node that needs to be replaced, and seamlessly connect to the computing link where the replaced node is located.

[0056] Although server 101 and master node 102 are shown as two separate devices in Figure 1A, it will be understood that in some embodiments, master node 102 may integrate the functions of server 101.

[0057] Figure 1B shows an exemplary projected image 1300 according to an embodiment of the present disclosure.

[0058] Projected image 1300 is, for example, a projected image (e.g., projected image 105 in Figure 1A) generated after a child node (e.g., child node 103-1 in Figure 1A) performs AI computational processing related to the AI ​​model training task. The AI ​​model training task includes both forward propagation computational processing and backward propagation computational processing. The AI ​​computational processing performed by this child node includes both AI computational processing for the forward propagation computational chain and AI computational processing for the backward propagation computational chain. Therefore, the intermediate execution result data of this child node includes both forward propagation data and backward propagation data.

[0059] As shown in Figure 1B, the projected image 1300 includes an edge region and a central region defined by three black positioning regions 1305. The edge region may include a forward propagation data type indicator region 1304 and a backward propagation data type indicator region 1310.

[0060] Each of the forward propagation data type indicator area 1304 and the backward propagation data type indicator area 1310 may include any one of the data type indicators 1311-1314.

[0061] Each of the data type indicators 1311-1314 may include a data type indication portion and a receiving node identifier portion. The data type indication portion indicates, in a pre-designed encoding format (e.g., a combination of blank and black areas), which data type is float16, int8, int32, or float32. The receiving node identifier portion indicates the identifier of the node receiving the corresponding data type. Taking data type indicator 1311 as an example, it includes a data type indication portion 1301 and a receiving node identifier portion 1302. The data type indication portion 1301 begins with a blank area, and the two consecutive blank areas following the blank area also indicate that the data type is float16. The receiving node identifier portion 1302 may be designed to contain a barcode (not shown) representing the identifier of the receiving node.

[0062] As shown in Figure 1B, the forward propagation data type indication area 1304 of the projected image 1300 includes an indicator designed as shown in 1311, i.e., the forward propagation data type is float16. The backward propagation data type indication area 1310 of the projected image 1300 also includes an indicator designed as shown in 1311, i.e., the backward propagation data type is also float16. However, it should be noted that the receiver node identifier indicated by the forward propagation data type indication area 1304 and the receiver node identifier indicated by the backward propagation data type indication area 1310 may be different.

[0063] The edge region of the projected image 1300 also includes a node status indicator 1307. The node status indicator 1307 includes, for example, three circular indicators: the first circle corresponds to an AI inference task, the second to a model training task, and the third to an online learning task. When the AI ​​computation of the corresponding task is executed normally, the corresponding circle is displayed in green; when the AI ​​computation of the corresponding task is executed abnormally, the corresponding circle is displayed in red; and when the corresponding task has been completed, the corresponding circle is displayed in blue. This node status indicator 1307 allows other child nodes and / or the main node to obtain relevant information by detecting the projected image.

[0064] Those skilled in the art will understand that the node status indicator 1307 can be designed in various ways as needed, as long as it can facilitate the transmission of various status information related to AI computing and processing at the child node.

[0065] The intermediate region of the projected image 1300 includes an intermediate execution result data region 1306. The intermediate execution result data region 1306 includes image data corresponding to the execution results of the AI ​​computational processing assigned to that child node. As shown in FIG1B, the intermediate execution result data region 1306 can be divided into multiple sub-regions 1308. For example, the overall projected image 1300 can be 4K (4096*2160) in size, and the intermediate execution result data region 1306 can include 24 VGA (640*480) sub-regions 1308. In some embodiments, these sub-regions 1308 can be associated with different AI computational processes. For example, as needed, 8 of the 24 sub-regions correspond to intermediate execution result data for AI computational processing used in the forward propagation computational chain, and the other 16 sub-regions correspond to intermediate execution result data for AI computational processing used in the backward propagation computational chain. In some embodiments, these sub-regions 1308 can also be used for different downstream nodes of the same AI computational process. Advantageously, the size of the sub-region 1308 can be determined based on the batch size of the AI ​​computational processing.

[0066] In the execution of AI model training tasks, the projected image designed according to the embodiments of this disclosure can support parallel forward propagation computation processing and backward propagation computation processing, which can effectively improve computational efficiency.

[0067] Those skilled in the art will understand that when a child node serves multiple different AI tasks, such as simultaneously serving AI model inference and AI model training tasks, parallel processing for both AI model inference and AI model training tasks can be supported. Based on the teachings of this disclosure, those skilled in the art can reasonably modify the format of the projected image as needed.

[0068] According to embodiments of this disclosure, information transmission between nodes is achieved via projected images, avoiding the limitations of traditional communication conditions (such as communication speed limitations, transmission errors, dependence on expensive transmission equipment, etc.). Transmitting information between nodes via projected images also allows intermediate results (projected images) to be stored in physical space, thus eliminating storage resource limitations for intermediate execution results.

[0069] Figure 2 shows an exemplary overall flowchart of a method 200 for a system 100 for distributed AI model training and / or inference according to embodiments of the present disclosure.

[0070] As shown in Figure 2, after method 200 begins, in step 201, the cloud (e.g., server 101) first determines whether it has received a task completion signal sent from the master node (e.g., master node 102). If it is determined to be yes in step 201, then method 200 ends.

[0071] If step 201 determines no, i.e., no task completion signal is received from the master node (e.g., master node 102), then method 200 continues to step 202. In step 202, the cloud determines whether it has received an instruction to replace the child node from the master node.

[0072] If the determination in step 202 is negative, then method 200 proceeds to step 203. In step 203, the cloud first determines whether the AI ​​task to be performed is a pre-trained inference task.

[0073] If the determination in step 203 is yes, then method 200 proceeds to step 206, whereby the cloud distributes the network structure or weight parameters of the pre-trained model corresponding to each sub-node for that stage to the relevant sub-nodes. The cloud can distribute device networking information and currently running network structure information to the master node. The cloud can also distribute at least a portion of the device networking information to the sub-nodes.

[0074] Then, method 200 proceeds to step 209, where each child node begins to execute the inference task.

[0075] Specifically, the master node initiates a synchronization signal, and upon receiving the signal, the child nodes begin inference operations. After detecting the projected image of the previous child node using an image sensor, each child node invokes the automatically adjusted auxiliary model of its internally pre-set image signal processor to perform image enhancement processing on the detected image. It then selects the image with the highest score corresponding to the intermediate execution result data area of ​​the previous child node, extracts the data from the intermediate result data area within that selected image, and performs the model inference calculation processing allocated to that child node on the extracted data to obtain the intermediate execution result data for that child node. Based on the calculated intermediate execution result data, the child node generates its own projected image and projects it onto a baffle or projection surface, allowing subsequent child nodes and the master node to obtain information related to the execution results of the child node's inference calculation processing by detecting this projected image.

[0076] In some embodiments, a verification process can be added to confirm that the information received by the child node is correct, thereby enhancing the reliability of the AI ​​system. For example, if the child node is not the initial child node of the computing link, it can extract the verification result (e.g., a checksum frame) of the intermediate execution result data frame of the previous child node from the synchronization frame sent by the master node, and calculate the verification result based on the intermediate execution result data detected by the child node from the projected image of the previous node. The two verification results are compared to determine whether there is an anomaly in the intermediate execution result of the child node. The synchronization frame can be transmitted via a wireless transmission method conceivable to those skilled in the art, such as via WiFi, Bluetooth, Zigbee, etc. If an anomaly exists, the child node can send an anomaly signal to the master node (e.g., via WiFi). Simultaneously, the child node can also include an indicator for indicating anomalies (e.g., the node status indicator 1307 shown in Figure 1A) on the image projected by the node, so that the master node and / or subsequent nodes can observe the anomaly by detecting the projected image.

[0077] In the absence of any abnormal child nodes, subsequent child nodes in the inference computation chain can repeat similar operations until all inference computation processing is completed (as shown in subsequent steps 214 and 215). The last child node sends the inference result and a task completion indication signal to the master node (e.g., via WiFi). The master node sends the inference result back to the cloud and broadcasts to all relevant child nodes that the inference task has been completed. Each child node maintains the format of the projected image and sets the status indicator corresponding to the completion of the inference task to the completed state (e.g., the first circle included in the node status indicator 1307 of Figure 1A is displayed in blue).

[0078] Returning to Figure 2, if the determination in step 203 is negative, meaning that the AI ​​task to be executed is not a pre-trained inference task, the process proceeds to step 204, where the cloud determines whether the AI ​​task to be executed is a training task (i.e., without using a pre-trained model, training the model from scratch).

[0079] If the determination is yes in step 204, then method 200 proceeds to step 207. In step 207, the cloud distributes the network structure of the model to each child node.

[0080] Then, method 200 proceeds to step 207, where each sub-node begins to execute the training task.

[0081] Specifically, the master node initiates a synchronization signal, and the child nodes, upon receiving the synchronization signal, begin the training operation. The forward propagation of the training process is similar to the operation combined with the inference process described in step 209.

[0082] However, backpropagation computation is performed after each forward propagation (inference) or simultaneously with forward propagation. In backpropagation, a child node detects the projected image of the next child node (e.g., child node 103-2 in Figure 1A detects the projected image of child node 103-3), updates its own network weights based on the detected projected image of the next child node, and performs AI computation processing (e.g., gradient calculation) related to backpropagation computation assigned to that child node, obtains intermediate execution result data, and projects the image for detection by the previous child node (e.g., child node 103-1) and / or the total node.

[0083] If no abnormal child nodes appear, repeat the above process until the calculation accuracy of the last child node meets the training requirements, at which point the task ends (as shown in subsequent steps 214 and 215). The last child node sends the accuracy result and task end flag back to the main node. The main node sends the accuracy result and task end flag back to the cloud, and simultaneously broadcasts the task execution to all child nodes. The projection format of each child node remains unchanged, and the node status indicator is changed to the end state (for example, the second circle included in the node status indicator 1307 in Figure 1A is displayed in blue).

[0084] As can be seen, in the model training task, forward propagation computation and backward propagation computation can be performed simultaneously, which can effectively improve computational efficiency.

[0085] Returning to Figure 2, if the determination in step 204 is negative, i.e., the AI ​​task to be performed is not a training task, then method 200 proceeds to step 205. In step 205, the cloud determines whether the AI ​​task to be performed is an online learning task.

[0086] If the determination is yes in step 205, then method 200 proceeds to step 208. In step 208, the cloud distributes the model's network structure and weight parameters to each child node, adjusts the network structure online, and sends the adjusted network structure to the backup node.

[0087] Next, method 200 proceeds to step 211. In step 211, each child node begins to execute the inference task, while the standby node begins to execute the training task.

[0088] Specifically, the master node initiates a synchronization signal, and the child nodes, upon receiving the signal, begin online learning. If the last two child nodes in the original computing link need to be replaced, two backup nodes are used to connect to the original computing link and begin executing the training task.

[0089] Repeat the process described in step 210 above until the previously erroneous inference results are corrected and the desired accuracy is achieved, at which point the task ends (as shown in subsequent steps 214 and 215). The backup node sends the accuracy result and task end flag back to the master node, which in turn sends the accuracy result and task end flag back to the cloud. Simultaneously, it broadcasts to all child nodes that the online learning task has ended. The projection format of each child node remains unchanged, and the node status indicator changes to the "end" state (e.g., the third circle included in node status indicator 1307 in Figure 1A is displayed in blue). The cloud re-networks, activating the two backup nodes for online learning and removing the last two replaced nodes. The cloud notifies the master node to restart the inference task using the computing link including the backup nodes. It is not necessary to reissue model weights and structure information; the remaining processes are the same as those described in conjunction with step 209.

[0090] In situations where an AI model cannot accurately identify certain new objects and needs to be updated without interrupting its normal operation, an online learning task can be performed. When online model correction is required, a backup node can be connected to replace the last two child nodes, and weight updates can be enabled, setting it to backpropagation mode until the new link can correctly identify the objects that were previously misinferred. At this point, weight updates of the backup node are disabled, the operating mode is changed to forward propagation, the original last two child nodes are removed, and the new link is enabled for inference.

[0091] In some embodiments, the master node may initiate the execution of an online learning task in response to receiving an online learning request.

[0092] Method 200 further includes step 212, whereby the master node determines whether there are any abnormal child nodes.

[0093] For example, the master node can receive anomaly signals from its child nodes, such as those sent by the corresponding child nodes to the master node when a verification inconsistency occurs during the execution of steps 209, 210, or 211. Alternatively, the master node can identify anomalous nodes, or even specific anomalous data, by detecting node status indicators in the projected images of each child node. The master node can also determine the presence of anomalous child nodes based on both the received anomaly signals and the detected node status indicators.

[0094] In some cases, the master node can determine that a child node has experienced a hardware failure, thus identifying that child node as an abnormal child node. In other cases, projected images can also indicate such anomalies caused by hardware failures, for example, through the abnormal status of node status indicators. The master node can also identify a child node and / or the previous child node (e.g., in the case of an error in the output received from the previous child node) as an abnormal or faulty node by error messages sent from the child node.

[0095] If the determination is yes in step 212, that is, if an abnormal child node is determined to exist, then method 200 proceeds to step 213, where the main node uploads the weight information of the child node to be replaced to the cloud. Method 200 then returns to step 202.

[0096] If step 202 determines that the child node needs to be replaced, then method 200 proceeds to step 216, where the cloud initializes a backup node and replaces the abnormal node with the backup node, and the task continues to be executed.

[0097] For example, upon detecting a projected image, the master node can first determine whether the abnormal projected image frame involves a training task (more specifically, whether it involves forward or backward propagation) or an inference task. It then sends a synchronization signal containing a request to send a message to the child node that projected the image. After receiving an anomaly feedback from that child node, the master node submits the anomaly feedback to the cloud. The cloud then allocates and initializes a new backup node, while simultaneously removing the replaced child node from the network.

[0098] For example, when online model correction is needed (e.g. in response to an online learning request), a backup node can be connected to replace the last two child nodes, and weight updates can be enabled and set to backpropagation mode until the new link can correctly identify the objects that were originally inferred incorrectly. At this point, the weight updates of the backup node are disabled, the working mode is changed to forward propagation, the original last two child nodes are removed, and the new link is enabled for inference.

[0099] It is understood that in the context of online learning, at least one child node at the end of the original computing link can be replaced, but it is not necessary to change all other child nodes of the original computing link. Those skilled in the art can determine the number of child nodes to be replaced as needed.

[0100] In online learning scenarios, training can be performed on only the last two child nodes using backup nodes, without halting the entire inference chain. That is, inference on the original chain and training (on the backup nodes) can occur simultaneously.

[0101] If it is determined in step 212 that there are no abnormal child nodes, then method 200 proceeds to step 214 to continue executing the inference task / training task / online learning task until step 215, when the main node detects that the task execution has ended, and then method 200 returns to step 201.

[0102] Figure 3 shows a flowchart of a method 300 performed by a master node according to an embodiment of the present disclosure.

[0103] As shown in Figure 3, after starting, method 300 proceeds to step 301, in which the master node determines whether a task end signal has been detected. This task end signal is, for example, sent to the master node by the last child node via WiFi.

[0104] If the determination in step 301 is yes, then method 300 proceeds to step 303, where the master node reads the final inference result or training state from the last child node. Then the method proceeds to step 304, where the master node sends the task completion information to the cloud and broadcasts the task execution completion to all child nodes. In response, each child node maintains its projection format and changes its node status indicator to the task completion state.

[0105] If the determination in step 301 is negative, i.e., no task end signal is detected, then method 300 proceeds to step 302, where the master node determines whether an anomaly has been detected in a child node. The master node can determine whether it has received an anomaly signal or error message from the child node or detected an anomaly status flag of the node status indicator in the image projected from the child node.

[0106] If the determination is yes in step 302, then method 300 proceeds to step 310, where the main node reads the specific exception information from the exception child node. Afterwards, method 300 proceeds to step 311, where the main node sends the exception child node identifier and the specific exception information to the cloud.

[0107] If the determination in step 302 is negative, i.e., no child node anomaly is detected, then method 300 proceeds to step 305. In step 305, the master node determines whether it has received the verification result (e.g., checksum frame) of the data frame from the child node.

[0108] If the determination in step 305 is negative, method 300 can proceed to step 306, where the master node determines whether the wait has timed out. If the master node does not receive the verification result of the data frame from a child node within the predetermined time period ("Yes" branch), it can determine that the child node is abnormal, and method 300 proceeds to step 309, where the master node sends the identifier of the child node that timed out and the timeout information to the cloud. If the master node receives the verification result of the data frame from a child node within the predetermined time period ("No" branch), then method 300 returns to step 305 and continues to wait.

[0109] If step 305 determines that the verification results of all child node data frames have been received, then method 300 proceeds to step 307, in which the master node loads the identifiers of all child nodes and the verification results of each child node's data frames into a synchronization frame. Then, method 300 proceeds to step 308, where the master node broadcasts the synchronization frame.

[0110] Figure 4 shows a flowchart of a method 400 performed by a child node according to an embodiment of the present disclosure.

[0111] As shown in Figure 4, after the method 400 starts, it proceeds to step 401 to determine whether a synchronization frame sent by the master node has been received.

[0112] If a synchronization frame is received in step 401, then method 400 proceeds to step 402, where the child node extracts the verification result of the data frame of the previous child node from the synchronization frame.

[0113] Then, method 400 proceeds to step 403, where the child node detects the projected image of the previous child node and identifies the receiving node identifier included in the projected image.

[0114] Method 400 continues to step 404, where the child node determines whether the identified receiving node identifier is consistent with the child node's own identifier.

[0115] If the identifier of the receiving node identified in step 404 is inconsistent with the identifier of the child node itself, then method 400 ends.

[0116] If the identifier of the receiving node identified by the sub-node in step 404 matches the identifier of the sub-node itself, then method 400 proceeds to step 405, where the sub-node detects the intermediate execution result data region in the projected image of the previous sub-node. Method 400 continues to step 406, where the sub-node can perform image enhancement on the extracted image data of the intermediate execution result data region and select the image frame with the highest score to obtain the corresponding data frame. Next, in step 407, the sub-node performs a verification calculation on the data frame and compares the calculated verification result of the data frame with the verification result extracted from the synchronization frame to check whether the verification results are consistent. If the verification is consistent, then method 400 continues to step 408, where the sub-node performs inference / training / online learning operations based on the data frame to obtain the execution result data frame of the sub-node. Then, in step 410, the sub-node calculates the verification result of the execution result data frame of the sub-node and sends the verification result to the summary point. In step 411, the sub-node projects the intermediate execution result image frame based on the execution result data frame of the sub-node.

[0117] If, in step 407, the child node determines that the calculated verification result is inconsistent with the verification result extracted from the synchronization frame, the method proceeds to step 409, whereby the child node sends an anomaly signal to the master node. In some embodiments, the master node can use the information carried by the anomaly signal, along with the state of the node status indicator detected in the projected image, to determine whether the node is abnormal and more specific anomaly information.

[0118] Figure 5 shows an example block diagram of an exemplary node device 500 according to an embodiment of the present disclosure. The node device 500 can be used as a child node or a backup node according to an embodiment of the present disclosure.

[0119] As shown in Figure 5, the node device 500 may include an image sensor 501, an image signal processor 502, and a projector 503. Although not shown, it will be understood that the node device may also include one or more other processors and / or memory.

[0120] Image sensors are responsible for converting light signals into electrical signals and are the core component of optical imaging systems. Image sensors can include, but are not limited to, CMOS sensors (Complementary Metal-Oxide-Semiconductor), CCD sensors (Charge-Coupled Device), and event-based cameras.

[0121] Image signal processors (ISPs) process the raw data output from image sensors to optimize image quality. ISPs can support AI enhancement features. Furthermore, they can perform AI computations at appropriate stages based on the network structure and weight parameters assigned to the node devices.

[0122] Projectors can encode information such as the computation results of node devices into light signals and project the encoded light signals onto a physical surface to form a projected image. Projectors can include, but are not limited to, projectors based on Digital Light Processing (DLP) technology.

[0123] Figure 6 shows an example flowchart of a method 600 performed by a child node device according to an embodiment of the present disclosure.

[0124] As shown in Figure 6, method 600 includes step 601, in which at least one AI computation process is performed.

[0125] Method 600 further includes step 603, in which an image is projected via a projector, the projected image being used to convey information relating to the execution result of the at least one AI computational process.

[0126] The projected image may include an execution result data region, which includes image data corresponding to the data that is the execution result of the at least one AI computation process. In some embodiments, the execution result data region includes sub-regions corresponding to each AI computation process in the at least one AI computation process.

[0127] At least one AI computational process may include multiple AI computational processes. In some embodiments, at least one first AI computational process among these multiple AI computational processes may be used for a first AI task, and at least one second AI computational process among these multiple AI computational processes may be used for a second AI task. In other embodiments, at least one third AI computational process among these multiple AI computational processes may be used for forward propagation computation of a third AI task, and at least one fourth AI computational process among these multiple AI computational processes may be used for backward propagation computation of a third AI task.

[0128] AI tasks can include, but are not limited to, AI model inference tasks, AI model training tasks, and AI model online learning tasks. The first, second, and third AI tasks here can be associated with the same AI model or with different AI models.

[0129] The projected image may further include a forward propagation indication region and a backward propagation indication region. The forward propagation indication region indicates the forward propagation data type and the corresponding first receiving node identifier, which indicates the identifier of the node device expected to detect the projected image in the forward propagation link. The backward propagation indication region indicates the backward propagation data type and the corresponding second receiving node identifier, which indicates the identifier of the node device expected to detect the projected image in the backward propagation link.

[0130] The projected image may also include node status indicators. Node status indicators may be used to indicate one of the following: the completion of the execution of the AI ​​task involved in at least one of the at least one AI computational processes; an execution error in at least one of the at least one AI computational processes; and normal execution of at least one of the at least one AI computational processes.

[0131] In some embodiments, method 600 may further include: detecting a projected image of an upstream node device to obtain a data frame of the upstream node device; performing the at least one AI computational process based on the data frame of the upstream node device to obtain a data frame of the node device; and projecting an image based on the obtained data frame of the node device for detection by a downstream node device. In some embodiments, detecting a projected image of an upstream node device to obtain a data frame of the upstream node device may include image enhancement of the detected projected image of the upstream node device, selecting the best image frame therefrom, and obtaining the corresponding data frame.

[0132] In some embodiments, method 600 may further include a verification operation. For example, method 600 may further include: extracting a verification frame of a data frame from an upstream node device from a synchronization frame from a master node; calculating a verification frame based on the data frame of the upstream node device obtained by detecting a projected image of the upstream node device; and comparing the calculated verification frame with the extracted verification frame to determine whether to send an abnormal signal to the master node.

[0133] Figure 7 shows an example block diagram of an exemplary node device 700 according to an embodiment of the present disclosure. The node device 700 can be used as a master node according to an embodiment of the present disclosure.

[0134] As shown in Figure 7, the node device 700 may include an image sensor 701 and an image signal processor 702. Although not shown, it will be understood that the node device may also include one or more other processors and / or memory.

[0135] Figure 8 shows an example flowchart of a method 800 performed by a node device according to an embodiment of the present disclosure. The node device can be used as a master node.

[0136] As shown in Figure 8, method 800 includes step 801, in which at least one child node device coupled to the node device is controlled to perform at least one AI task, each AI task including at least one corresponding AI computation process.

[0137] Method 800 further includes step 803, in which a projected image of each child node device is detected via an image sensor to receive information relating to the execution result of at least one AI computational process assigned to that child node device.

[0138] In some embodiments, method 800 may further include: determining that a child node is an abnormal node, such as a faulty child node, based on at least one of an abnormal signal received from the child node device and the detection of a node status indicator in the projected image of the child node; initializing a backup node and synchronizing the backup node for the faulty child node; and replacing the faulty child node with the backup node to seamlessly access the computing link where the faulty child node is located.

[0139] In some embodiments, method 800 may further include: in response to an online learning request, determining that at least one child node device at the end of the original computing link needs to be replaced. In this case, method 800 may further include: for the at least one child node device at the end of the original computing link, connecting at least one backup node device in the original computing link and completing online learning in a backpropagation state until a new computing link including the at least one backup node device repairs the error of the original computing link, and replacing the at least one child node device at the end of the original computing link with the at least one backup node device.

[0140] In some embodiments, method 800 may further include: receiving a verification frame from the at least one child node device, generating a synchronization frame based at least on the received verification frame from the at least one child node device, and broadcasting the synchronization frame.

[0141] In some embodiments, method 800 may further include: during the initialization phase, forming a network for the AI ​​task to be executed and sending network information to the at least one child node; and during the operation phase, adjusting the network structure based on node anomaly information and backup node information.

[0142] Figure 9 illustrates a flowchart of a method 900 for training and / or inferring a distributed AI model according to an embodiment of the present disclosure. The system for training and / or inferring a distributed AI model according to an embodiment of the present disclosure may include a master node (e.g., master node 102 shown in Figure 1A or node device 600 shown in Figure 6) and at least one child node coupled to the master node (e.g., child nodes 103-1 to 103-n shown in Figure 1A or node device 500 shown in Figure 5).

[0143] As shown in Figure 9, method 900 may include step 901, in which a child node transmits information related to the execution result of at least one AI computation process assigned to the child node to other child nodes and / or to the master node through a projected image.

[0144] In some embodiments, the system for training and / or inference of a distributed AI model according to embodiments of this disclosure may further include a server. In this case, method 900 may further include the server, during the initialization phase, forming a network for the AI ​​task to be performed and issuing network information to the master node and the at least one child node, and during the operation phase, adjusting the network structure based on node anomaly information and backup node information fed back by the master node.

[0145] It is understandable that the server's functions can be integrated into the master node; therefore, the master node can also perform the operations that were performed by the server.

[0146] Those skilled in the art will understand that Figures 5 and 7 merely illustrate some exemplary unit configurations of a node device (including a main node, child nodes, and / or standby nodes). Those skilled in the art can implement various corresponding units based on the operations to be performed by the node device. More specifically, in general, a node device may include a processor (processing circuitry) and a memory (not shown). The memory stores computer-readable instructions containing corresponding instructions for various operations to be performed by the node device. For example, for a node device according to embodiments of this disclosure, the computer-readable instructions may include, but are not limited to, instructions for performing operations performed by the node device as shown in Figures 2-4, 6, 8, and 9. The processor can implement the functions of units corresponding to these operations by reading and executing the corresponding instructions stored in the memory for various operations to be performed by the base station.

[0147] The functions of the various elements disclosed herein can be implemented using circuitry or processing circuitry including general-purpose processors, application-specific processors, integrated circuits, AISC (“Application-Specific Integrated Circuits”), conventional circuits, and / or combinations thereof, which are configured or programmed to perform the disclosed functions. A processor is considered a circuit or processing circuit because it includes transistors and other circuitry. In this disclosure, a circuit, unit, or component is hardware that performs or is programmed to perform the described functions. The hardware can be any hardware disclosed herein or otherwise known that is programmed or configured to perform the described functions. When the hardware is a processor (which is considered a circuit), the circuit, unit, or component is a combination of hardware and software used to configure the hardware and / or the processor.

[0148] It should be understood that the functions according to embodiments of this disclosure can also be implemented by software and / or firmware. In the case of implementation by software and / or firmware, programs constituting the software are installed from a storage medium or network onto a computer with a dedicated hardware architecture, such as the general-purpose computer 1100 shown in FIG. 10. This computer, when various programs are installed, is capable of performing various functions, etc. FIG. 10 is a block diagram illustrating an example structure of a computer with an optimized device that can be employed in embodiments of this disclosure.

[0149] In one example, the computer may correspond to the exemplary monitoring device described above according to embodiments of the present disclosure. At least some modules included in the detection device according to embodiments of the present disclosure may be implemented by running associated program instructions from the central processing unit of the general-purpose computer 1100 shown in FIG. 10.

[0150] In Figure 10, the Central Processing Unit (CPU) 1101 performs various processes according to the program stored in the Read-Only Memory (ROM) 1102 or the program loaded from the storage section 1108 into the Random Access Memory (RAM) 1103. The RAM 1103 also stores, as needed, the data required when the CPU 1101 performs various processes, etc.

[0151] It is understood that in some embodiments, the memory may be integrated into the central processing unit.

[0152] CPU 1101, ROM 1102 and RAM 1103 are connected to each other via bus 1104. Input / output interface 1105 is also connected to bus 1104.

[0153] The following components are connected to the input / output interface 1105: input section 1106, including a keyboard, mouse, etc.; output section 1107, including a display, such as a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; storage section 1108, including a hard disk, etc.; and communication section 1109, including a network interface card, such as a LAN card, modem, etc. The communication section 1109 performs communication processing via a network, such as the Internet.

[0154] As needed, drive 1110 is also connected to input / output interface 1105. Removable media 1111, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 1110 as needed, so that computer programs read from them can be installed into storage section 1108 as needed.

[0155] When the above series of processes are implemented through software, the program constituting the software is installed from a network such as the Internet or a storage medium such as removable media 1111.

[0156] Those skilled in the art will understand that such storage media are not limited to the removable medium 1111 shown in FIG. 10, which stores programs and is distributed separately from the device to provide programs to users. Examples of removable media 1111 include magnetic disks (including floppy disks (registered trademark)), optical disks (including optical disc read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini-disk (MD) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be ROM 1102, a hard disk included in storage section 1108, etc., which stores programs and is distributed to users along with the device containing them.

[0157] It should be noted that the methods and devices described herein can be implemented as software, firmware, hardware, or any combination thereof. Some components may be implemented, for example, as software running on a digital signal processor or microprocessor. Other components may be implemented, for example, as hardware and / or application-specific integrated circuits (ASICs).

[0158] Furthermore, the methods and systems of the present invention can be implemented in various ways. For example, the methods and systems of the present invention can be implemented by software, hardware, firmware, or any combination thereof. The order of steps of the method described above is merely illustrative, and unless otherwise specifically stated, the steps of the method of the present invention are not limited to the order specifically described above. In addition, in some embodiments, the present invention can also be embodied as a program recorded in a recording medium, including machine-readable instructions for implementing the method according to the present invention. Therefore, the present invention also covers recording media storing programs for implementing the method according to the present invention. Such storage media may include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, etc.

[0159] Those skilled in the art will recognize that the boundaries between the above operations are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed among additional operations, and operations may be performed with at least partial overlap in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments. However, other modifications, variations, and substitutions are equally possible. Therefore, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

[0160] [Exemplary Implementation of this Disclosure]

[0161] Based on the embodiments of this disclosure, various implementations of the concepts of this disclosure are conceivable, including but not limited to:

[0162] 1. A system for training and / or inferring distributed artificial intelligence (AI) models, comprising:

[0163] The main node includes the image sensor and the image signal processor; and

[0164] At least one child node coupled to the main node, each child node including an image sensor, an image signal processor, and a projector.

[0165] The child node is configured to transmit information related to the execution result of at least one AI computation process assigned to it to other child nodes and / or to the master node by projecting an image.

[0166] 2. The system as described in item 1, wherein the projected image of the child node includes:

[0167] The execution result data area includes image data corresponding to the execution result of the at least one AI computational process assigned to the child node.

[0168] 3. The system as described in item 2, wherein the execution result data area includes sub-regions corresponding to each AI computing process in the at least one AI computing process.

[0169] 4. The system as described in item 1, wherein at least one AI computational process assigned to a child node comprises multiple AI computational processes, wherein,

[0170] (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or

[0171] (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

[0172] 5. The system as described in item 1, wherein the projected image comprises:

[0173] A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the child node expected to be detected in the projected image in the forward propagation link; and

[0174] The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the child node of the projected image to be detected in the backpropagation link.

[0175] 6. The system as described in item 1, wherein the projected image of the child node further includes:

[0176] A node status indicator that indicates one of the following:

[0177] (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes assigned to the child node has ended;

[0178] (ii) An execution exception occurs in at least one of the at least one AI computation processes assigned to the child node; and

[0179] (iii) At least one of the AI ​​computation processes assigned to the child node is executed normally.

[0180] 7. The system as described in item 1, wherein the child nodes are configured as follows:

[0181] The data frames of the upstream child nodes are obtained by detecting the projected images of the upstream child nodes.

[0182] To obtain the data frame of a child node, at least one AI computational process is performed based on the data frame of the upstream child node.

[0183] The image is projected based on the data frame obtained from the child node for downstream child node detection.

[0184] 8. The system as described in item 7, wherein the child nodes are configured as follows:

[0185] Extract the verification frame of the data frame from the upstream child node from the synchronization frame from the master node;

[0186] The verification frame is calculated based on the data frame of the upstream child node obtained by detecting the projected image of the upstream child node;

[0187] The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

[0188] 9. The system as described in item 1, wherein the master node is configured to control the at least one child node to execute at least one AI task, each AI task including at least one corresponding AI computational process, and the AI ​​task including at least one of the following: AI model inference task, AI model training task, or AI model online learning task.

[0189] 10. The system as described in item 1, wherein the master node is configured as follows:

[0190] A child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node and the detection of the projected image of the child node.

[0191] 11. The system as described in item 10, wherein determining a child node as an abnormal node includes determining a child node as a faulty child node, and the master node is configured as follows:

[0192] For the failed child node, initialize the standby node and complete the synchronization of the standby node;

[0193] Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

[0194] 12. The system as described in item 1, wherein the master node is configured as follows:

[0195] In response to an online learning request, it is determined that at least one child node at the end of the original computing link needs to be replaced.

[0196] 13. The system as described in item 12, wherein the master node is configured as follows:

[0197] For the at least one child node at the end of the original computing link, at least one backup node is connected in the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node repairs the error of the original computing link;

[0198] Replace the at least one child node at the end of the original computing link with the at least one spare node.

[0199] 14. The system as described in item 1, wherein the master node is configured as follows:

[0200] Receive the verification frame from at least one of the child nodes;

[0201] A synchronization frame is generated based at least on the received verification frame of the at least one child node; and

[0202] Broadcast synchronization frame.

[0203] 15. The system as described in item 1 further includes a server, wherein the server is configured to perform at least one of the following operations:

[0204] During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to the master node and at least one child node; and

[0205] During the operation phase, the network structure is adjusted based on the node anomaly information and backup node information fed back by the main node.

[0206] 16. The system as described in item 15, wherein the master node includes the server.

[0207] 17. A method performed by a system for training and / or inference of a distributed AI model, the system comprising a master node and at least one child node coupled to the master node, the master node comprising an image sensor and an image signal processor, and each child node comprising an image sensor, an image signal processor, and a projector, wherein the method comprises:

[0208] The child node transmits information related to the execution result of at least one AI computation process assigned to it to other child nodes and / or to the master node by projecting an image.

[0209] 18. The method as described in item 17, wherein the projected image of the child node includes:

[0210] The execution result data area includes image data corresponding to the execution result of the at least one AI computational process assigned to the child node.

[0211] 19. The method as described in item 17, wherein the execution result data area includes sub-regions corresponding to each AI computing process in the at least one AI computing process.

[0212] 20. The method of claim 17, wherein at least one AI computational process assigned to a child node comprises multiple AI computational processes, wherein,

[0213] (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or

[0214] (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

[0215] 21. The method as described in item 17, wherein the projected image comprises:

[0216] A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the child node expected to be detected in the projected image in the forward propagation link; and

[0217] The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the child node of the projected image to be detected in the backpropagation link.

[0218] 22. The method as described in item 17, wherein the projected image of the child node further includes:

[0219] A node status indicator that indicates one of the following:

[0220] (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes assigned to the child node has ended;

[0221] (ii) An execution exception occurs in at least one of the at least one AI computation processes assigned to the child node; and

[0222] (iii) At least one of the AI ​​computation processes assigned to the child node is executed normally.

[0223] 23. The method described in item 17 further includes child nodes:

[0224] The data frames of the upstream child nodes are obtained by detecting the projected images of the upstream child nodes.

[0225] To obtain the data frame of a child node, at least one AI computational process assigned to that child node is performed based on the data frame of the upstream child node; and

[0226] The image is projected based on the data frame obtained from the child node for downstream child node detection.

[0227] 24. The method described in item 23 further includes child nodes:

[0228] Extract the verification frame of the data frame from the upstream child node from the synchronization frame from the master node;

[0229] The verification frame is calculated based on the data frame of the upstream child node obtained by detecting the projected image of the upstream child node;

[0230] The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

[0231] 25. The method of item 17 further includes controlling the at least one sub-node to execute at least one AI task by the master node, each AI task including at least one corresponding AI computational process, the AI ​​task including at least one of the following: AI model inference task, AI model training task, or AI model online learning task.

[0232] 26. The method described in item 17 further includes a master node:

[0233] A child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node and the detection of the projected image of the child node.

[0234] 27. The method as described in item 26, wherein determining a child node as an abnormal node includes determining a child node as a faulty child node, and the method further includes determination by the parent node:

[0235] For the failed child node, initialize the standby node and complete the synchronization of the standby node;

[0236] Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

[0237] 28. The method described in item 17 further includes a master node:

[0238] In response to an online learning request, it is determined that at least one child node at the end of the original computing link needs to be replaced.

[0239] 29. The method described in item 28, further comprising a master node:

[0240] For at least one child node at the end of the original computing link, at least one backup node is connected in the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node repairs the error of the original computing link.

[0241] Replace the at least one child node at the end of the original computing link with the at least one backup node.

[0242] 30. The method described in item 17 further includes a master node:

[0243] Receive the verification frame from at least one of the child nodes;

[0244] A synchronization frame is generated based at least on the received verification frame of the at least one child node; and

[0245] Broadcast synchronization frame.

[0246] 31. The method described in item 17 further includes a server:

[0247] During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to the master node and at least one child node; and

[0248] During the operation phase, the network structure is adjusted based on the node anomaly information and backup node information fed back by the main node.

[0249] 32. A computer program medium having stored computer-executable instructions thereon, which, when executed by a processor, cause the methods described in items 17-31 to be performed.

[0250] 33. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the methods described in items 17-31 to be performed.

[0251] 34. A method executed by a node device, comprising:

[0252] Perform at least one AI computational process; and

[0253] An image is projected via a projector, and the projected image is used to convey information related to the execution result of the at least one AI computational process.

[0254] 35. The method as described in item 34, wherein the projected image comprises:

[0255] The execution result data area includes image data corresponding to the data that is the execution result of the at least one AI calculation process.

[0256] 36. The method as described in item 34, wherein the execution result data area includes sub-regions corresponding to each AI computing process in the at least one AI computing process.

[0257] 39. The method as described in item 34, wherein the at least one AI computational process comprises multiple AI computational processes, wherein,

[0258] (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or

[0259] (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

[0260] 37. The method as described in item 34, wherein the projected image comprises:

[0261] A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the node device expected to detect the projected image in the forward propagation link; and

[0262] The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the node device expected to detect the projected image in the backpropagation link.

[0263] 38. The method as described in item 34, wherein the projected image further includes:

[0264] A node status indicator that indicates one of the following:

[0265] (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes has ended;

[0266] (ii) An execution exception occurs in at least one of the at least one AI computation processes; and

[0267] (iii) At least one of the at least one AI computational processes is executed normally.

[0268] 40. The method described in item 34 further includes:

[0269] The data frames of the upstream node device are obtained by detecting the projected image of the upstream node device.

[0270] To obtain the data frame of the node device, at least one AI computational process is performed based on the data frame of the upstream node device; and

[0271] The obtained data frames from the node device are used to project images for detection by downstream node devices.

[0272] 41. The method described in item 40 further includes:

[0273] Extract the verification frame of the data frame from the upstream node device from the synchronization frame from the master node;

[0274] Based on the data frames obtained from the projection images of the upstream node devices, a verification frame is calculated; and

[0275] The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

[0276] 42. A node device comprising an image sensor, an image signal processor, and a projector, wherein the node device is configured to perform the method as described in any one of items 34-41.

[0277] 43. A computer program medium having stored computer-executable instructions thereon, which, when executed by a processor, cause the methods described in items 34-41 to be performed.

[0278] 44. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the methods described in items 34-41 to be performed.

[0279] 45. A method executed by a node device, comprising:

[0280] Control at least one child node device coupled to the node device to execute at least one AI task, each AI task including at least one corresponding AI computation process;

[0281] The projected image of each child node device is detected via an image sensor to receive information related to the execution result of at least one AI computational process assigned to that child node device.

[0282] 46. ​​The method of claim 45, wherein detecting the projected image of each child node device includes detecting a node status indicator included in the projected image, wherein the node status indicator is used to indicate one of the following:

[0283] (i) The execution of the AI ​​task involved in at least one of the at least one AI computing processes assigned to the sub-node device has ended;

[0284] (ii) An execution exception of at least one of the at least one AI computing processes assigned to the child node device; and

[0285] (iii) At least one of the at least one AI computing processes assigned to the sub-node device is executing normally.

[0286] 47. The method described in item 46 further includes:

[0287] The child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node device and the detection of the node status indicator in the projected image of the child node device.

[0288] 48. The method described in item 47, wherein determining that a child node device is an abnormal node includes determining that the child node device is a faulty child node, the method comprising:

[0289] For the failed child node, initialize the standby node and complete its synchronization; and

[0290] Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

[0291] 49. The method described in item 46 further includes:

[0292] In response to an online learning request, it is determined that at least one child node device at the end of the original computing link needs to be replaced.

[0293] 50. The method described in item 49 further includes:

[0294] For the at least one child node device at the end of the original computing link, at least one backup node device is connected to the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node device repairs the error of the original computing link.

[0295] Replace the at least one child node device at the end of the original computing link with the at least one backup node.

[0296] 51. The method described in item 45 further includes:

[0297] Receive the verification frame from the at least one child node device;

[0298] A synchronization frame is generated based at least on the verification frame received from the at least one child node device; and

[0299] Broadcast synchronization frame.

[0300] 52. The method described in item 45 further includes:

[0301] During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to at least one child node device; and

[0302] During the operation phase, the network structure is adjusted based on node anomaly information and backup node information.

[0303] 53. The method as described in item 45, wherein the AI ​​task includes at least one of the following: an AI model inference task, an AI model training task, or an AI model online learning task.

[0304] 54. A node device including an image sensor and an image signal processor, wherein the node device is configured to perform the method as described in any one of items 45-53.

[0305] 55. A computer program medium having stored computer-executable instructions thereon, which, when executed by a processor, cause the methods described in items 45-53 to be performed.

[0306] 56. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the methods described in items 45-53 to be performed.

Claims

1. A system for training and / or inferring distributed artificial intelligence (AI) models, comprising: The main node includes the image sensor and the image signal processor; and At least one child node coupled to the main node, each child node including an image sensor, an image signal processor, and a projector. The child node is configured to transmit information related to the execution result of at least one AI computation process assigned to it to other child nodes and / or to the master node by projecting an image.

2. The system as claimed in claim 1, wherein, The projected images of the child nodes include: The execution result data area includes image data corresponding to the execution result of the at least one AI computational process assigned to the child node.

3. The system as described in claim 2, wherein, The execution result data area includes sub-regions corresponding to each AI computation process in the at least one AI computation process.

4. The system as claimed in claim 1, wherein, At least one AI computational process assigned to a child node includes multiple AI computational processes, wherein, (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

5. The system as claimed in claim 1, wherein, The projected image includes: A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the child node expected to be detected in the projected image in the forward propagation link; and The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the child node of the projected image to be detected in the backpropagation link.

6. The system as claimed in claim 1, wherein, The projected images of the child nodes also include: A node status indicator that indicates one of the following: (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes assigned to the child node has ended; (ii) An execution exception occurs in at least one of the at least one AI computation processes assigned to the child node; and (iii) At least one of the AI ​​computation processes assigned to the child node is executed normally.

7. The system as claimed in claim 1, wherein, The child node is configured as follows: The data frames of the upstream child nodes are obtained by detecting the projected images of the upstream child nodes. To obtain the data frame of a child node, at least one AI computational process is performed based on the data frame of the upstream child node. The image is projected based on the data frame obtained from the child node for downstream child node detection.

8. The system of claim 7, wherein, The child node is configured as follows: Extract the verification frame of the data frame from the upstream child node from the synchronization frame from the master node; The verification frame is calculated based on the data frame of the upstream child node obtained by detecting the projected image of the upstream child node; The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

9. The system of claim 1, wherein, The master node is configured to control the at least one child node to execute at least one AI task, each AI task including at least one corresponding AI computation process, and the AI ​​task including at least one of the following: AI model inference task, AI model training task, or AI model online learning task.

10. The system of claim 1, wherein, The master node is configured as follows: A child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node and the detection of the projected image of the child node.

11. The system of claim 10, wherein, Determining a child node as an abnormal node includes determining a child node as a faulty child node. The main node is configured as follows: For the failed child node, initialize the standby node and complete the synchronization of the standby node; Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

12. The system of claim 1, wherein, The master node is configured as follows: In response to an online learning request, it is determined that at least one child node at the end of the original computing link needs to be replaced.

13. The system of claim 12, wherein, The master node is configured as follows: For the at least one child node at the end of the original computing link, at least one backup node is connected in the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node repairs the error of the original computing link; Replace the at least one child node at the end of the original computing link with the at least one spare node.

14. The system of claim 1, wherein, The master node is configured as follows: Receive the verification frame from at least one of the child nodes; A synchronization frame is generated based at least on the received verification frame of the at least one child node; and Broadcast synchronization frame.

15. The system of claim 1, further comprising a server, wherein, The server is configured to perform at least one of the following operations: During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to the master node and at least one child node; and During the operation phase, the network structure is adjusted based on the node anomaly information and backup node information fed back by the main node.

16. The system of claim 15, wherein, The master node includes the server.

17. A method performed by a system for training and / or inference of a distributed AI model, the system comprising a master node and at least one child node coupled to the master node, the master node comprising an image sensor and an image signal processor, and each child node comprising an image sensor, an image signal processor, and a projector, wherein the method comprises: The child node transmits information related to the execution result of at least one AI computation process assigned to it to other child nodes and / or to the master node by projecting an image.

18. The method of claim 17, wherein, The projected images of the child nodes include: The execution result data area includes image data corresponding to the execution result of the at least one AI computational process assigned to the child node.

19. The method of claim 17, wherein, The execution result data area includes sub-regions corresponding to each AI computation process in the at least one AI computation process.

20. The method of claim 17, wherein, At least one AI computational process assigned to a child node includes multiple AI computational processes, wherein, (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

21. The method of claim 17, wherein, The projected image includes: A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the child node expected to be detected in the projected image in the forward propagation link; and The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the child node of the projected image to be detected in the backpropagation link.

22. The method of claim 17, wherein, The projected images of the child nodes also include: A node status indicator that indicates one of the following: (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes assigned to the child node has ended; (ii) An execution exception occurs in at least one of the at least one AI computation processes assigned to the child node; and (iii) At least one of the AI ​​computation processes assigned to the child node is executed normally.

23. The method of claim 17, further comprising child nodes: The data frames of the upstream child nodes are obtained by detecting the projected images of the upstream child nodes. To obtain the data frame of a child node, at least one AI computational process assigned to that child node is performed based on the data frame of the upstream child node; and The image is projected based on the data frame obtained from the child node for downstream child node detection.

24. The method of claim 23, further comprising child nodes: Extract the verification frame of the data frame from the upstream child node from the synchronization frame from the master node; The verification frame is calculated based on the data frame of the upstream child node obtained by detecting the projected image of the upstream child node; The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

25. The method of claim 17, further comprising controlling the at least one sub-node to execute at least one AI task by the master node, each AI task including at least one corresponding AI computational process, the AI ​​task including at least one of the following: AI model inference task, AI model training task, or AI model online learning task.

26. The method of claim 17, further comprising a master node: A child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node and the detection of the projected image of the child node.

27. The method of claim 26, wherein, Determining a child node as an abnormal node includes determining that the child node is a faulty child node. The method also includes determination by the parent node: For the failed child node, initialize the standby node and complete the synchronization of the standby node; Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

28. The method of claim 17, further comprising a master node: In response to an online learning request, it is determined that at least one child node at the end of the original computing link needs to be replaced.

29. The method of claim 28, wherein, It also includes the total node: For at least one child node at the end of the original computing link, at least one backup node is connected in the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node repairs the error of the original computing link. Replace the at least one child node at the end of the original computing link with the at least one backup node.

30. The method of claim 17, further comprising a master node: Receive the verification frame from at least one of the child nodes; A synchronization frame is generated based at least on the received verification frame of the at least one child node; and Broadcast synchronization frame.

31. The method of claim 17, further comprising a server: During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to the master node and at least one child node; and During the operation phase, the network structure is adjusted based on the node anomaly information and backup node information fed back by the main node.

32. A computer program medium having stored thereon computer-executable instructions, which, when executed by a processor, cause the method of claims 17-31 to be performed.

33. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the method of claims 17-31 to be performed.

34. A method executed by a node device, comprising: Perform at least one AI computational process; and An image is projected via a projector, and the projected image is used to convey information related to the execution result of the at least one AI computational process.

35. The method of claim 34, wherein, The projected images include: The execution result data area includes image data corresponding to the data that is the execution result of the at least one AI calculation process.

36. The method of claim 34, wherein, The execution result data area includes sub-regions corresponding to each AI computation process in the at least one AI computation process.

37. The method of claim 34, wherein, The at least one AI computational process includes multiple AI computational processes, wherein, (i) at least one first AI computational process among the plurality of AI computational processes is used for a first AI task, and at least one second AI computational process among the plurality of AI computational processes is used for a second AI task; and / or (ii) At least one third AI computation process among the plurality of AI computation processes is used for forward propagation computation of the third AI task and at least one fourth AI computation process among the plurality of AI computation processes is used for backward propagation computation of the third AI task.

38. The method of claim 34, wherein, The projected images include: A forward propagation indication area is used to indicate the forward propagation data type and the corresponding first receiving node identifier, wherein the first receiving node identifier indicates the identifier of the node device expected to detect the projected image in the forward propagation link; and The backpropagation indication area is used to indicate the backpropagation data type and the corresponding second receiving node identifier, which indicates the identifier of the node device expected to detect the projected image in the backpropagation link.

39. The method of claim 34, wherein, The projected image also includes: A node status indicator that indicates one of the following: (i) The execution of the AI ​​task involved in at least one of the at least one AI computational processes has ended; (ii) An execution exception occurs in at least one of the at least one AI computation processes; and (iii) At least one of the at least one AI computational processes is executed normally.

40. The method of claim 34, further comprising: The data frames of the upstream node device are obtained by detecting the projected image of the upstream node device. The at least one AI computational process is performed based on the data frame of the upstream node device to obtain the data frame of the node device; and The obtained data frames from the node device are used to project images for detection by downstream node devices.

41. The method of claim 40, further comprising: Extract the verification frame of the data frame from the upstream node device from the synchronization frame from the master node; The verification frame is calculated based on the data frame of the upstream node device obtained by detecting the projected image of the upstream node device. and The calculated check frame is compared with the extracted check frame to determine whether an abnormal signal should be sent to the master node.

42. A node device comprising an image sensor, an image signal processor, and a projector, wherein the node device is configured to perform the method as claimed in any one of claims 34-41.

43. A computer program medium having stored thereon computer-executable instructions, which, when executed by a processor, cause the method of any one of claims 34-41 to be performed.

44. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the method of any one of claims 34-41 to be performed.

45. A method executed by a node device, comprising: Control at least one child node device coupled to the node device to execute at least one AI task, each AI task including at least one corresponding AI computation process; The projected image of each child node device is detected via an image sensor to receive information related to the execution result of at least one AI computational process assigned to that child node device.

46. ​​The method of claim 45, wherein, Detecting the projected image of each child node device includes detecting node status indicators included in the projected image, wherein the node status indicators are used to indicate one of the following: (i) The execution of the AI ​​task involved in at least one of the at least one AI computing processes assigned to the sub-node device has ended; (ii) An execution exception of at least one of the at least one AI computing processes assigned to the child node device; and (iii) At least one of the at least one AI computing processes assigned to the sub-node device is executing normally.

47. The method of claim 46, further comprising: The child node is determined to be an abnormal node based on at least one of the abnormal signal received from the child node device and the detection of the node status indicator in the projected image of the child node device.

48. The method of claim 47, wherein determining that a child node device is an abnormal node includes determining that the child node device is a faulty child node, the method comprising: For the failed child node, initialize the standby node and complete the synchronization of the standby node; and Replace the faulty child node with a backup node to seamlessly connect to the computing link where the faulty child node is located.

49. The method of claim 46, further comprising: In response to an online learning request, it is determined that at least one child node device at the end of the original computing link needs to be replaced.

50. The method of claim 49, further comprising: For the at least one child node device at the end of the original computing link, at least one backup node device is connected to the original computing link and online learning is completed in the back propagation state until the new computing link including the at least one backup node device repairs the error of the original computing link. Replace the at least one child node device at the end of the original computing link with the at least one backup node.

51. The method of claim 45, further comprising: Receive the verification frame from the at least one child node device; A synchronization frame is generated based at least on the verification frame received from the at least one child node device; and Broadcast synchronization frame.

52. The method of claim 45, further comprising: During the initialization phase, a network is formed for the AI ​​task to be executed, and network information is sent to at least one child node device; and During the operation phase, the network structure is adjusted based on node anomaly information and backup node information.

53. The method of claim 45, wherein, AI tasks include at least one of the following: AI model inference tasks, AI model training tasks, or AI model online learning tasks.

54. A node device including an image sensor and an image signal processor, wherein the node device is configured to perform the method as claimed in any one of claims 45-53.

55. A computer program medium having stored thereon computer-executable instructions, which, when executed by a processor, cause the method of any one of claims 45-53 to be performed.

56. A computer program product comprising computer-executable instructions, which, when executed by a processor, cause the method of any one of claims 45-53 to be performed.

Images