Data processing network, data processing method and related device

Abstract

Provided in the embodiments of the present application are a data processing network, a data processing method and a related device, which are used for achieving computing power synergy between different intelligent devices. The data processing network provided in the embodiments of the present application comprises: a computing power demand module, a computing power management module and at least one computing execution module. The computing power demand module is used for issuing an execution request for a target task. The computing power management module is used for acquiring the resource condition of the at least one computing execution device and, on the basis of the resource condition of the at least one computing execution module, allocating to the at least one computing execution module sub-tasks of the target task to be executed. The at least one computing execution module is used for processing the sub-tasks allocated by the computing power management module and feeding processing results of the sub-tasks back to the computing power management module. The computing power management module is further used for integrating the processing results of the subtasks into a processing result of the target task, and feeding the processing result of the target task back to the computing power demand module.

Description

A data processing network, a data processing method, and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202410507881.X, filed on April 24, 2024, entitled “A Data Processing Network, Data Processing Method and Related Equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing, and more particularly to a data processing network, a data processing method, and related equipment. Background Technology

[0003] A smart home integrates technologies such as the Internet of Things, cloud computing, mobile internet, and big data to effectively combine intelligent control of home devices, home environment sensing, family health sensing, home security sensing, information exchange, and consumer services to create a healthy, safe, comfortable, low-carbon, and convenient personalized home life.

[0004] In smart home solutions, some smart devices need to perform tasks (such as identifying features from video and audio data). Performing these tasks consumes the computing power of the smart devices. As smart home solutions are upgraded, the computing power required for these tasks increases significantly, and current smart devices may not be able to handle the ever-increasing computing power demands.

[0005] One solution is to upgrade the hardware of smart devices to increase computing power, but hardware upgrades are costly. Another solution is to upload tasks to the cloud, use the cloud's computing power to execute the tasks, and return the processing results to the smart devices. However, this method carries the risk of data leakage (for example, video data uploaded to the cloud for processing may be leaked during data transmission). Summary of the Invention

[0006] This application provides a data processing network, a data processing method, and related equipment to enable computing power collaboration among different smart devices and meet the diverse smart demands of smart home scenarios.

[0007] In a first aspect, embodiments of this application provide a data processing network. This data processing network includes: a computing power demand module (driver), a computing power management module (master), and at least one computing execution module (worker). The computing power management module and the at least one computing execution module are deployed on different devices. The computing power demand module is used to issue execution requests for a target task. The computing power management module is used to obtain the resource status of at least one computing execution device and, based on the resource status of the at least one computing execution module, allocate subtasks in the target task to the at least one computing execution module for execution. The at least one computing execution module is used to process the subtasks allocated by the computing power management module and feed back the processing results of the subtasks to the computing power management module. The computing power management module is also used to integrate the processing results of the subtasks into the processing result of the target task and feed back the processing result of the target task to the computing power demand module.

[0008] In this embodiment, different computing execution modules (workers) with computing resources can inform the computing power management module (master) of their resource status. The computing power management module (master) then assigns the target task to the computing execution module (worker) with matching resources for processing. This method provides a unified resource management and invocation protocol for different computing execution devices (workers, i.e., devices with computing resources), enabling the computing power management module (master) to manage the computing power of different devices and achieve computing power sharing among different devices, allocating computing power to the computing power-demanding modules (drivers) that need it. This achieves computing power collaboration between different smart devices, meeting the diverse intelligent demands of smart home scenarios.

[0009] In one alternative implementation, both the computing power requirement module and the computing power management module are deployed on the first device, and at least one computing execution module is deployed on at least one second device; or, the computing power management module is deployed on the first device, the computing power requirement module is deployed on the third device, and at least one computing execution module is deployed on at least one second device.

[0010] In this embodiment, the computing power requirement module and the computing power management module can be deployed on the same device or on different devices, improving the flexibility of the network architecture. The computing power management module is typically deployed on a device located at the core of the home network. The computing power requirement module can be deployed on the device at the core location or on devices in other locations within the home network, ensuring that all locations within the home network can be included in the network provided in this embodiment, thereby completing the target tasks published at each location.

[0011] In one alternative implementation, the computing power management module is also used to send the execution status of the target task to the computing power demand module.

[0012] In this embodiment, the computing power management module (master) acts as the core for allocating computing power resources and can determine the execution progress and status of the target task. After the computing power demand module (driver) publishes the target task to the computing power management module (master), the master informs the driver of the execution status of the target task, enabling the driver to promptly obtain information about the execution status of the target task.

[0013] In one alternative implementation, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0014] In this embodiment, the execution status can reflect whether the target task was successful (success / failure) or the execution progress of the target task. If the target task has been completed, the execution status can include the processing result of the target task.

[0015] In one alternative implementation, the computing power requirement module is also used to query the computing power management module for the execution status of the target task.

[0016] Optionally, the computing power management module master can also proactively send the execution status of the target task to the computing power demand module driver; this application does not limit this.

[0017] In one alternative implementation, the computing power management module is further configured to: send at least one configuration information to at least one computing execution module; and the at least one computing execution module is further configured to: perform initialization based on at least one configuration information.

[0018] In this embodiment, the initialization of the computing execution module is achieved through configuration information from the computing power management module. This allows the computing power management module and the computing execution module to align their initial states, improving the reliability of the solution.

[0019] In one alternative implementation, at least one computation execution module is further configured to: send a keep-alive message to the computing power management module; the computing power management module is further configured to: send a keep-alive response message to at least one computation execution module.

[0020] In this embodiment, keep-alive messages and keep-alive response messages ensure that the link connection between the computing execution module and the computing power management module is normal.

[0021] In one alternative implementation, the computing power management module is further configured to: query the current resource status from at least one computing execution module; and at least one computing execution module is further configured to: reply to the computing power management module with the current resource status.

[0022] In this embodiment, the computing power management module can adaptively adjust the subtasks allocated to the execution module based on the current resource status of the computing execution module, ensuring the completion effect of the subtasks and thereby improving the completion effect of the target task.

[0023] In one alternative implementation, the computing power management module is also used to: send a deregistration instruction to the target computing execution module in at least one computing execution module.

[0024] In this embodiment of the application, if the computing power management module determines that it does not need to use the computing power of a certain computing execution module, or if a certain computing execution module is about to be taken offline from the data processing network, the unregistration of the computing execution module can be achieved by using the unregistration instruction.

[0025] In one alternative implementation, the resource configuration includes at least one of the following: a central processing unit (CPU), a neural processing unit (NPU), storage, and double data rate synchronous dynamic random access memory (DDR).

[0026] Secondly, embodiments of this application provide a data processing method. This method is applied to a first device in a network (the first device deploys a computing power management module, master). The network also includes at least one second device (the second device deploys computing execution devices, worker). The method includes: receiving at least one first message from the at least one second device, each first message corresponding to one of the at least one second device, the first message including resource information of the corresponding second device; obtaining an execution request for a target task; sending a second message to a target device among the at least one second device, the second message instructing the target device to process a target subtask within the target task, the target subtask being allocated according to the target device's resources; and receiving a reporting message from the target device, the reporting message including the processing result of the target subtask.

[0027] The data processing network and data processing method provided in this application are a low-cost computing power collaboration solution. They can not only meet the diverse intelligent needs of smart home scenarios, but also enable interconnection and interoperability of home Internet of Things (IoT) devices (i.e., different smart devices), improving the user experience.

[0028] This solution requires no hardware upgrades and is relatively low-cost. Furthermore, it can assign the target task to devices within the FTTR network (such as FTTR master devices, slave devices, or smart devices as secondary devices), allowing the task to be completed within the home network. It eliminates the need to upload home information to the cloud for processing, thus protecting family privacy and reducing the risk of data breaches.

[0029] In one optional implementation, the step of obtaining the execution request of the target task includes: receiving a third message from a third device (deploying a computing power requirement module driver), the third message being used to request processing of the target task; after receiving the reported message from the target device, the method further includes: sending a fourth message to the third device, the fourth message including the processing result of the target task, the processing result of the target task being obtained by integrating the processing results of subtasks reported by different second devices.

[0030] In one alternative implementation, after sending a second message to the target device in at least one second device, the method further includes sending a task status to a third device, the task status indicating the execution status of the target task.

[0031] In one alternative implementation, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0032] In one alternative implementation, before sending the task status to the third device, the method further includes: receiving a query request from the third device, the query request being used to query the execution status of the target task.

[0033] In one alternative implementation, after receiving at least one first message from at least one second device and before obtaining the execution request for the target task, the method further includes: sending at least one configuration message to at least one second device, wherein the at least one configuration message corresponds one-to-one with at least one second device, and the configuration message includes configuration information of the corresponding second device, the configuration information being used to initialize the second device.

[0034] In one optional implementation, the method further includes: receiving a keep-alive message from the at least one second device; and sending a keep-alive response message to the at least one second device.

[0035] In one alternative implementation, the method further includes: sending a query message to at least one second device, the query message being used to query the current resource status of the second device; and receiving a query response message from at least one second device, the query response message being used to indicate the current resource status of the second device.

[0036] In one alternative implementation, the method further includes sending a deregistration instruction to the target device in at least one second device.

[0037] In one alternative implementation, the resource configuration includes at least one of the following: CPU, NPU, storage, and DDR memory.

[0038] Thirdly, embodiments of this application provide a data processing method. This method is applied to a second device in a network (the second device deploys a computing execution device worker). The network also includes a first device (the first device deploys a computing power management module master). The method includes: sending a first message to the first device, the first message including the resource status of the second device; receiving a second message from the first device, the second message instructing the second device to process a target subtask in a target task, the target subtask being allocated according to the resources of the second device; and sending a reporting message to the first device, the reporting message including the processing result of the target subtask.

[0039] In one alternative implementation, after sending the first message to the first device, the method further includes: receiving a configuration message from the first device, the configuration message including configuration information of the second device; and initializing the second device according to the configuration information.

[0040] In one alternative implementation, the method further includes: sending a keep-alive message to the first device; and receiving a keep-alive response message from the first device.

[0041] In one alternative implementation, the method further includes: receiving a query message from a first device, the query message being used to query the current resource status of a second device; and sending a query response message to the first device, the query response message being used to indicate the current resource status of the second device.

[0042] In one alternative implementation, the method further includes receiving a deregistration instruction from the first device.

[0043] In one alternative implementation, the resource configuration includes at least one of the following: CPU, NPU, storage, and DDR memory.

[0044] Fourthly, embodiments of this application provide a data processing method. This method is applied to a third device in a network (the third device deploys a computing power demand module, driver). The network also includes a first device (the first device deploys a computing power management module, master) and at least one second device (the second device deploys a computing execution device, worker). The method includes: sending a third message to the first device, the third message being used to request processing of a target task; receiving a fourth message from the first device, the fourth message including the processing result of the target task, the processing result of the target task being an integration of the processing results of sub-tasks reported by different second devices, where the sub-tasks processed by different second devices are all included in the target task.

[0045] In one alternative implementation, after sending the third message to the first device, the method further includes: receiving a task status from the first device, the task status indicating the execution status of the target task.

[0046] In one alternative implementation, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0047] In one alternative implementation, before receiving the task status from the first device, the method further includes: sending a query request to the first device, the query request being used to query the execution status of the target task.

[0048] Fifthly, embodiments of this application provide a first device. The first device is included in a network, which also includes at least one second device. The first device includes a sending unit and a receiving unit. The receiving unit is configured to: receive at least one first message from the at least one second device, each first message corresponding to one of the at least one second device, the first message including resource information of the corresponding second device; and obtain an execution request for a target task. The sending unit is configured to: send a second message to a target device among the at least one second device, the second message instructing the target device to process a target subtask within the target task, the target subtask being allocated according to the resources of the target device. The receiving unit is further configured to: receive a reporting message from the target device, the reporting message including the processing result of the target subtask.

[0049] In one optional implementation, the receiving unit is further configured to: receive a third message from a third device (the deployment computing power requirement module driver), the third message being used to request processing of the target task. The sending unit is further configured to: send a fourth message to the third device, the fourth message including the processing result of the target task, the processing result of the target task being obtained by integrating the processing results of subtasks reported by different second devices.

[0050] In an optional implementation, the sending unit is further configured to: send a task status to a third device, the task status indicating the execution status of the target task.

[0051] In one alternative implementation, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0052] In one optional implementation, the receiving unit is further configured to: receive a query request from a third device, the query request being used to query the execution status of the target task.

[0053] In one optional implementation, the sending unit is further configured to: send at least one configuration message to at least one second device, wherein the at least one configuration message corresponds one-to-one with at least one second device, and the configuration message includes configuration information of the corresponding second device, the configuration information being used to initialize the second device.

[0054] In one optional implementation, the receiving unit is further configured to: receive a keep-alive message from the at least one second device. The sending unit is further configured to: send a keep-alive response message to the at least one second device.

[0055] In one optional implementation, the sending unit is further configured to: send a query message to at least one second device, the query message being used to query the current resource status of the second device. The receiving unit is further configured to: receive a query response message from at least one second device, the query response message being used to indicate the current resource status of the second device.

[0056] In one alternative implementation, the sending unit is further configured to: send a deregistration instruction to the target device in at least one second device.

[0057] In one alternative implementation, the resource configuration includes at least one of the following: CPU, NPU, storage, and DDR memory.

[0058] Sixthly, embodiments of this application provide a second device. The second device is included in a network, which also includes a first device. The second device includes a sending unit and a receiving unit. The sending unit is configured to: send a first message to the first device, the first message including the resource status of the second device. The receiving unit is configured to: receive a second message from the first device, the second message instructing the second device to process a target sub-task in a target task, the target sub-task being allocated according to the resources of the second device. The sending unit is further configured to: send a reporting message to the first device, the reporting message including the processing result of the target sub-task.

[0059] In one optional implementation, the receiving unit is further configured to: receive a configuration message from the first device, the configuration message including configuration information of the second device; and initialize the second device according to the configuration information.

[0060] In one optional implementation, the sending unit is further configured to: send a keep-alive message to the first device. The receiving unit is further configured to: receive a keep-alive response message from the first device.

[0061] In one optional implementation, the receiving unit is further configured to: receive a query message from the first device, the query message being used to query the current resource status of the second device. The sending unit is further configured to: send a query response message to the first device, the query response message being used to indicate the current resource status of the second device.

[0062] In one alternative implementation, the receiving unit is further configured to: receive a deregistration instruction from the first device.

[0063] In one alternative implementation, the resource configuration includes at least one of the following: CPU, NPU, storage, and DDR memory.

[0064] In a seventh aspect, embodiments of this application provide a third device. This third device is included in a network, which also includes a first device and at least one second device. The third device includes a sending unit and a receiving unit. The sending unit is configured to: send a third message to the first device, the third message being a request to process a target task. The receiving unit is configured to: receive a fourth message from the first device, the fourth message including the processing result of the target task, the processing result of the target task being an integration of the processing results of sub-tasks reported by different second devices, where the sub-tasks processed by different second devices are all included in the target task.

[0065] In one alternative implementation, the receiving unit is further configured to: receive a task status from the first device, the task status indicating the execution status of the target task.

[0066] In one alternative implementation, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0067] In one alternative implementation, the sending unit is further configured to: send a query request to the first device, the query request being used to query the execution status of the target task.

[0068] Eighthly, embodiments of this application provide an intelligent device. The intelligent device includes a processor and interface circuitry. Optionally, it also includes a memory. The processor is coupled to the memory and the interface circuitry.

[0069] The interface circuitry is used to receive signals from devices other than the smart device and transmit them to the processor, or to send signals from the processor to devices other than the smart device. The memory is used to store computer programs or instructions.

[0070] When the processor executes the computer program or instructions, it causes the communication device to perform the method executed by the first device in the above method embodiments.

[0071] Alternatively, when the processor executes the computer program or instructions, the communication device may execute the method performed by the second device in the above method embodiments.

[0072] Alternatively, when the processor executes the computer program or instructions, the communication device may execute the method performed by the third device in the above method embodiments.

[0073] Ninthly, embodiments of this application provide a computer-readable storage medium. This computer-readable storage medium stores a program that, when executed by a computer, performs the methods described in the second aspect or any possible implementation thereof, the third aspect or any possible implementation thereof, or the fourth aspect or any possible implementation thereof.

[0074] Tenthly, embodiments of this application provide a computer program product. When the computer program product is executed on a computer, the computer performs the methods described in the second aspect or any possible implementation of the second aspect, the third aspect or any possible implementation of the third aspect, or the fourth aspect or any possible implementation of the fourth aspect.

[0075] The beneficial effects of aspects three through ten are described in aspects one and two, and will not be repeated here. Attached Figure Description

[0076] Figure 1 is a schematic diagram of an architecture of a data processing network provided in an embodiment of this application;

[0077] Figure 2 is a schematic diagram of an architecture of the distributed computing power platform provided in an embodiment of this application;

[0078] Figure 3 is a schematic diagram of the connection relationship of the data processing network provided in the embodiment of this application;

[0079] Figure 4 is a schematic diagram of the protocol interface of the data processing network provided in the embodiment of this application;

[0080] Figure 5 is a flowchart illustrating a data processing method provided in an embodiment of this application.

[0081] Figure 6a is a schematic diagram of a data processing network provided in an embodiment of this application;

[0082] Figure 6b is another schematic diagram of the data processing network provided in an embodiment of this application;

[0083] Figure 7 is a flowchart illustrating the computing power management stage of the data processing method provided in the embodiments of this application;

[0084] Figure 8 is a flowchart illustrating the computing power allocation stage of the data processing method provided in the embodiments of this application;

[0085] Figure 9 is a structural schematic diagram of a smart device provided in an embodiment of this application;

[0086] Figure 10 is a structural schematic diagram of a first device provided in an embodiment of this application;

[0087] Figure 11 is a structural schematic diagram of a second device provided in an embodiment of this application;

[0088] Figure 12 is a structural schematic diagram of a third device provided in an embodiment of this application. Detailed Implementation

[0089] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0090] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. Additionally, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be expressed as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0091] In the field of smart homes, smart devices are used to perform intelligent tasks. For example, smart cameras can perform tasks such as identifying features like people and pets from video data, and smart lights can perform tasks such as automatically turning on based on room brightness and time of day.

[0092] As smart home solutions are upgraded, the computing power required for tasks performed by smart devices is increasing, and current smart devices may not be able to handle the ever-increasing computing power consumption.

[0093] For example, smart speakers were originally designed to enable intelligent dialogue with users. With the upgrade of smart home solutions, smart speakers are required to recognize the voiceprints of different users and adopt different dialogue modes for different users (e.g., an elderly dialogue mode for elderly users and a toddler dialogue mode for toddlers). Because voiceprint recognition requires significant computing power, current smart speakers lack the computing power to support it, making the upgrade solution unusable for smart speakers.

[0094] There are currently two main solutions. One is to upgrade the hardware, replacing the smart devices. However, hardware upgrades are costly. The other approach is to upload the task to the cloud for execution, but there is a risk of information leakage during the cloud upload process.

[0095] For example, smart speakers upload users' voice data to the cloud for voiceprint recognition, and the cloud then sends the recognition results back to the smart device. However, there is a risk of voice data leakage during the transmission of voice data from the smart speaker to the cloud.

[0096] To address the aforementioned issues, embodiments of this application provide a data processing network, a data processing method, and related equipment. The data processing network provided in these embodiments offers a unified resource management and access protocol for different intelligent devices, thereby enabling collaborative management and access of computing power across different intelligent devices.

[0097] As shown in Figure 1, the data processing network provided in this embodiment includes: a computing power demand module (driver), a computing power management module (master), and at least one computing execution module (worker).

[0098] The network has one and only one master computing power management module, while the number of driver computing power demand modules and worker computing execution modules can be multiple.

[0099] In one alternative implementation, the computing power management module (master) and the computing execution module (worker) are deployed on different devices. The computing power demand module (driver) and the computing power management module (master) can be deployed on the same device or on different devices.

[0100] Optionally, the computing power management module (master) and the computing execution module (worker) can also be deployed on the same device; this application does not impose any restrictions on this.

[0101] In one example, the computing power management module (master) can be deployed on the fiber-to-the-room (FTTR) main device, such as on the main FTTR unit (MFU) or access controller (AC). Leveraging the central position of the FTTR master device in the smart home, it enables collaborative management and control of the computing power of various smart devices within the home's smart scenario.

[0102] The computing power requirement module (driver) can be deployed on any smart device, FTTR master device, or FTTR slave device (e.g., a sub-FTTR unit (SFU), wireless access point (AP), etc.). The computing execution module (worker) can be deployed on any device with computing power resources, such as a smart device, FTTR master device, or FTTR slave device; this application does not impose any restrictions on this.

[0103] In this embodiment, the computing power management module master has the functions of computing power management and computing power allocation. Computing power management refers to the management of computing power resources of multiple computing execution module workers in the network; computing power allocation refers to the assignment of tasks from the computing power management module master to one or more computing execution module workers for execution.

[0104] Specifically, the computing power management module master can obtain the resource status of at least one computing execution device worker, thereby realizing centralized management of the computing power resources of multiple computing execution module workers.

[0105] If a smart device in the network receives the execution instruction for a target task, the computing power demand module (driver) deployed on that smart device can send an execution request for the target task to the computing power management module (master). The master can then allocate subtasks within the target task to at least one computing execution module (worker) for execution, based on the resource availability of at least one computing execution module (worker). Each computing execution module (worker) can process the subtasks allocated by the master and return the processing results to the master. The master then integrates the subtask processing results into the target task's processing result and returns it to the computing power demand module (driver). This process enables the allocation of computing resources among smart devices.

[0106] In this application embodiment, the resource configuration includes at least one of the following: a central processing unit (CPU), a neural network processing unit (NPU), storage, and double-rate synchronous dynamic random access memory (DDR memory).

[0107] In the network shown in Figure 1, there is typically one master computing power management module, usually deployed at the core of a home network (e.g., the FTTR master device in an FTTR network). The network can include one or more driver computing power requirement modules and one or more worker computing execution modules. The driver computing power requirement module can be deployed on the same device as the master computing power management module, or on different devices. The worker computing execution module can be deployed on the same device as the master computing power management module, or on different devices.

[0108] In this embodiment, the data processing network is also referred to as a distributed computing (collaborative) foundation. As shown in Figure 2, upper-layer services such as intelligent storage, intelligent sensing, and intelligent interaction can be implemented through the distributed computing foundation. Specifically, the driver is used to run the above services and generate corresponding target tasks. The distributed computing foundation includes an FTTR master device (i.e., the master FTTR in Figure 2), FTTR slave devices (i.e., the slave FTTR in Figure 2), and intelligent devices such as cameras.

[0109] The Master can be deployed on the main FTTR device to manage the computing power of various devices in the distributed computing power base, that is, to manage the computing power of the FTTR master device, FTTR slave devices, and smart devices such as cameras. If the upper layer application issues a target task (i.e., the driver issues a target task), the Master on the main FTTR device can aggregate the computing power of the FTTR master device, FTTR slave devices, and third-party devices (smart devices, etc.) for use by the business layer.

[0110] Optionally, the service layer may include FTTR's own services, such as FTTR-NAS home subscribers and applications from third-party devices, such as home cameras.

[0111] This application embodiment also provides a unified protocol interface for each device in the distributed computing power base. As shown in Figure 3, the driver provides a distributed programming interface (API) to the application layer, through which the application layer submits computing tasks to the manager. The manager is responsible for managing all devices under the distributed computing power base, such as cameras and set-top boxes. The driver publishes the target task to the manager through the protocol interface, and the manager distributes the target subtasks to the worker devices for execution through the protocol interface. The worker executes the target subtasks issued by the manager and feeds back the processing results of the target subtasks to the manager through the protocol interface.

[0112] It is worth noting that the protocol interfaces between the driver and the manager, and between the manager and the worker, can be used to send messages with specific content, such as the first message and the second message mentioned below.

[0113] It is worth noting that the protocol interfaces between the driver and manager, and between the manager and worker, are decoupled from the network layer, transport layer, and link layer protocols, making them adaptable to various networking scenarios. For example, they can connect to a remote control via Bluetooth as shown in Figure 4, or to a camera via Wi-Fi.

[0114] Optionally, the driver can also send the execution status of the target task to the master via a protocol interface. Prior to this, the driver can query the master for the execution status of the target task. The execution status can include at least one of the following: success, failure, execution progress, and processing result.

[0115] Optionally, the master can also proactively send the execution status of the target task to the driver; this application does not limit this.

[0116] Optionally, the master can also send at least one configuration message to at least one worker via a protocol interface; at least one worker can initialize itself based on the configuration message. This achieves state alignment between the master and worker.

[0117] Optionally, at least one worker can also send a keep-alive message to the computing power management module, and the master sends a keep-alive response message to at least one worker. This ensures that the link connection between the master and the workers is normal.

[0118] Optionally, the master can also query the current resource status from at least one worker via a protocol interface, and at least one worker will reply with the current resource status to the master. This allows the master to adaptively adjust the subtasks assigned to the workers based on their current resource status, ensuring the effective completion of the subtasks and thus improving the overall performance of the target task.

[0119] Optionally, the master can also send a deregistration command to at least one target worker, thereby deregistering the target worker in the network.

[0120] Based on the networks shown in Figures 1 to 3, this application provides a method for processing computing resources, as shown in Figure 5. The method includes:

[0121] 501. At least one second device sends a first message to the first device, the first message including the resource status of the corresponding second device.

[0122] In this embodiment, the first device is a device with a master deployed. Optionally, the first device can be an FTTR master device, so that the master is at the core of the home network.

[0123] The network includes at least one worker, which can be deployed on different second devices. Optionally, the second device can be a smart device, an FTTR slave device, an FTTR master device, etc., and this application does not limit it.

[0124] The second device can send a first message to the first device, which is used to indicate the resource status of the workers deployed on the second device.

[0125] After obtaining the resource status of each worker through the first message, the first device can record the resource status of each worker and allocate sub-tasks to each worker according to the (computing power) resource status of each worker when a task needs to be executed.

[0126] It is worth noting that, in addition to the FTTR master device, the first device can also be other types of devices, such as smart devices like smart speakers, or FTTR slave devices, etc., and this application does not limit them.

[0127] 502. The first device obtains the execution request of the target task.

[0128] Optionally, in addition to the master, a driver can also be deployed on the first device. For example, as shown in Figure 6a, a master, driver, and worker are deployed on the first device (FTTR master device). When the driver wants to execute a target task, the driver sends an execution request for the target task to the master, and the first device can then obtain the execution request for the target task.

[0129] Optionally, if the master that publishes the target task is deployed on a third device outside the first device, the third device can send a third message to the first device. The third message is used to request processing of the target task. For example, as shown in Figure 6b, the third device is a camera, the driver is deployed on the camera, the master is deployed on the FTTR master device, and the workers are deployed on the camera, the FTTR master device, and the FTTR slave device.

[0130] 503. The first device sends a second message to the target device in at least one of the second devices. The second message is used to instruct the target device to process the target sub-task in the target task. The target sub-task is based on the resource allocation of the target computing power device.

[0131] The first device can divide the target task into one or more target subtasks, and assign the subtasks to appropriate workers for execution based on the resource availability of each worker.

[0132] Specifically, after the first device divides the target subtasks, it determines the workers from multiple workers to execute the target subtasks and sends a second message to these workers. The second message instructs the second device, upon receiving the message, to execute the corresponding target subtask.

[0133] 504. The target device sends a reporting message to the first device. The reporting message includes the processing result of the target subtask.

[0134] After the worker (located on the target device) assigned to the target subtask completes its execution, it transmits the processing result of the target subtask to the first device via a reporting message. The master on the first device can then integrate the processing results of the target subtasks into the processing result of the target task and report the processing result of the target task back to the driver.

[0135] Optionally, if the master that publishes the target task is deployed on a third device outside the first device, the first device can send a fourth message to the third device. The fourth message includes the processing result of the target task.

[0136] In this embodiment, different workers (deployed on the second device) with computing resources can inform the master (deployed on the first device) of their resource status via a first message. The master (first device) then assigns the target task to a worker (second device) with matching resources for processing. This method provides a unified resource management and invocation protocol for different computing execution devices (workers, i.e., devices with computing resources), enabling the master to manage the computing power of different devices and achieve sharing of computing power among different devices, allocating computing power to the required driver devices. This achieves computing power collaboration between different smart devices, meeting the diverse intelligent demands of smart home scenarios.

[0137] The data processing network and data processing method provided in this application are a low-cost computing power collaboration solution. They can not only meet the diverse intelligent needs of smart home scenarios, but also enable interconnection and interoperability of home Internet of Things (IoT) devices (i.e., different smart devices), improving the user experience.

[0138] This solution requires no hardware upgrades and has a low cost. Furthermore, it distributes the target task to devices within the FTTR network (such as FTTR master devices, slave devices, or smart devices), allowing the task to be completed within the home network. It eliminates the need to upload home information to the cloud for processing, protecting family privacy and reducing the risk of data breaches.

[0139] In this embodiment of the application, in order to allocate and execute the target task within the home network, two stages are required: computing power management and computing power allocation, which will be described in detail below:

[0140] I. The stage of integrating computing power into the management system.

[0141] As shown in Figure 7, the computing power management phase includes:

[0142] 701. The worker sends the first message to the master, reporting the resource status.

[0143] The computing power management phase mainly includes the corresponding processes of workers and masters. For ease of description, this application embodiment uses the FTTR master device and slave device (e.g., SFU or AP) as examples of the first and second devices. That is, the master is deployed on the FTTR master device, and the worker is deployed on the FTTR slave device.

[0144] During the initialization phase, the worker (deployed on the FTTR slave device, i.e., the second device) can send the first message to the master (deployed on the FTTR master device, i.e., the first device), registering the computing resources of the FTTR slave device to the FTTR master device through the first message.

[0145] Optionally, the first message may include the computing power of the FTTR slave device (e.g., CPU / NPU / encoding / decoding, etc.) and the size of the computing power.

[0146] Optionally, the first message can be a Register frame or other types of messages, and this application does not limit this.

[0147] 702. Identifiers issued by the master to the worker.

[0148] After receiving the first message from the FTTR slave device, the FTTR master device can assign an identifier to the FTTR slave device, which is used to indicate different workers in the network.

[0149] Optionally, this identifier can be an Agent ID. The FTTR master device can notify the FTTR slave device of this identifier via a Register Ack frame.

[0150] 703. The master sends a configuration message to the worker, which includes the worker's configuration information.

[0151] The FTTR master device can send configuration messages to the FTTR slave device. The configuration message includes the configuration information of the FTTR slave device, which is used to initialize the FTTR slave device.

[0152] Optionally, configuration information may include the number of compute instances of FTTR slave devices, keep-alive period, etc.

[0153] Optionally, the configuration message can be an Agent Config frame.

[0154] 704. The worker sends a response message to the master when sending a configuration message.

[0155] After receiving the configuration message from the FTTR master device, the FTTR slave device initializes its own related functions according to the configuration information and sends a response message to the FTTR master device. Optionally, this response message can be an Agent Config Ack frame.

[0156] Through the above steps, the FTTR master device has completed the management of the computing power of the FTTR slave devices. That is, the computing power management module master has obtained the resource status of at least one computing execution device worker, and can realize centralized management of the computing power resources of multiple computing execution module workers.

[0157] Optionally, third-party devices such as home set-top boxes and smart TVs can also access the data processing network (also known as a distributed computing power base) provided in this application embodiment based on the above interaction process.

[0158] 705. The worker sends a keep-alive message to the master.

[0159] FTTR slave devices can send keep-alive messages to FTTR master devices according to the keep-alive cycle.

[0160] 706. The master replies to the worker with a keep-alive response message.

[0161] After receiving the keep-alive message, the FTTR master device sends a keep-alive response message (Keep Live Ack message) to the FTTR slave device to ensure that the link connection between the master FTTR device and the slave FTTR device is normal.

[0162] 707. The master sends a query message to the worker.

[0163] After computing power is managed, the FTTR master device can send query messages to the FTTR slave device. The query message is used to query the current resource status of the second device. Optionally, the query message can be a Query frame, used to query the remaining computing power resources of the worker.

[0164] 708. The worker sends a query response message to the master.

[0165] The FTTR slave device sends a query response message to the FTTR master device, providing feedback on the current resource status of the FTTR slave device.

[0166] 709. The master sends a registration message to the worker.

[0167] After computing power is managed, the FTTR master device can select to remove a specific FTTR from the distributed computing power base based on the network topology. The FTTR master device then sends a deregistration message to the FTTR slave device it decides to remove. Optionally, the deregistration message can be an UnRegister frame.

[0168] Optionally, the deregistration message may include the Agent ID of the FTTR slave device to be removed from the distributed computing power base, so that the FTTR slave device with that Agent ID can determine to deregister.

[0169] 710. The worker sends a registration response message to the master.

[0170] After completing the deregistration process, the FTTR slave device sends a deregistration response message to the FTTR master device.

[0171] In this embodiment of the application, IoT devices can connect to the FTTR distributed computing power base based on the above protocol process to achieve centralized management of computing power resources.

[0172] II. Computing power allocation phase.

[0173] As shown in Figure 8, the computing power allocation phase includes:

[0174] 801. The driver sends a third message to the master.

[0175] Take a network attached storage (NAS) scenario as an example. When a user uploads photos to the FTTR-NAS, the NAS smart album application (driver) can send a third message to the FTTR master device. This third message is used to request processing of a target task (such as classifying images).

[0176] Optionally, the third message can be a Job Commit frame.

[0177] 802. The master sends a response message to the driver in response to a third message.

[0178] The FTTR master device responds to the NAS smart photo album application with a Job Commit Ack frame.

[0179] 803. The master sends a second message to the worker.

[0180] During the computing power management phase, the Master obtains the computing power resource information of each worker. Therefore, based on the resource status of at least one computing execution module worker, the Master can allocate subtasks in the target task to at least one worker (referred to as the target device) for execution and send a second message to the target device. The second message instructs the target device to process the target subtasks in the target task, and the target subtasks are allocated according to the resource allocation of the target device.

[0181] Optionally, the second message can be a Job Dispatch frame.

[0182] 804. The worker sends a response message to the master for the second message.

[0183] After receiving the computing task, the Worker module responds to the Master module via the Job Dispatch Ack frame and starts the computing task to execute the assigned target subtask.

[0184] 805. The worker sends a keep-alive message to the master.

[0185] During the execution of the target subtask, the Worker sends keep-alive messages (such as Keep Live frames) to the Master to maintain a heartbeat and keep the worker alive.

[0186] Optionally, the Worker can report the status of the target subtask in real time via heartbeat frames until the calculation is completed.

[0187] 806. The master sends a keep-alive response message to the worker.

[0188] Optionally, the keep-alive response message can be a Keep Live Ack frame.

[0189] 807. The master sends the fourth message to the driver.

[0190] After the Master detects that the target subtask has been completed, it integrates the processing results of the subtasks reported by different second devices (workers) into the processing result of the target task. Then, it reports the processing result of the target task to the NAS smart photo album application via a fourth message.

[0191] Optionally, the fourth message can be a Job Complete frame.

[0192] 808. The driver sends a query request to the master.

[0193] During the execution of the target task, the NAS Smart Album application (driver) can also send query requests (such as Query commands) to the master. Query requests are used to check the execution status of the target task.

[0194] 809. The master sends the task status to the driver.

[0195] The Master sends the execution status of the target task to the driver task status.

[0196] Optionally, the execution status includes at least one of the following: success, failure, execution progress, and processing result.

[0197] 810. The driver sends a termination command to the master.

[0198] If the driver wants to terminate the execution of the target task, it can send a termination command (such as the Job Stop command) to the master.

[0199] 811. The master sends a termination command to the driver.

[0200] The Master sends a termination command (e.g., a Job Stop command) to the driver (target device) executing the target subtask.

[0201] 812. The driver sends a termination response (e.g., Job Stop Ack) to the master.

[0202] After receiving the termination command, the driver sends a termination response to the master and terminates the execution of the target subtask.

[0203] 813. The master sends a termination response to the driver.

[0204] After receiving the termination response from the target device, the master sends a termination response (e.g., Job Stop Ack) to the driver.

[0205] In this embodiment, computing power is allocated to the driver devices that require it through computing power allocation. This enables computing power collaboration between different smart devices, meeting the diverse intelligent needs of smart home scenarios.

[0206] The above describes the computing resource processing method provided in the embodiments of this application. The following describes the device used to implement the method:

[0207] This application also provides a smart device. As shown in FIG9, the smart device 900 includes a processor 910 and an interface circuit 920. The processor 910 and the interface circuit 920 are coupled to each other. It is understood that the interface circuit 920 can be a transceiver or an input / output interface.

[0208] Optionally, the smart device 900 may also include a memory 930 for storing instructions executed by the processor 910, or storing input data required by the processor 910 to execute instructions, or storing data generated after the processor 910 executes instructions.

[0209] When the aforementioned smart device 900 is the first device in the embodiments shown in Figures 5 to 8 or a chip applied to the first device, the smart device 900 implements the functions of the first device in the above method embodiments.

[0210] When the aforementioned smart device 900 is the second device in the embodiments shown in Figures 5 to 8 or a chip applied to the second device, the smart device 900 implements the functions of the second device in the above method embodiments.

[0211] When the aforementioned smart device 900 is the third device in the embodiments shown in Figures 5 to 8 or a chip applied to the third device, the smart device 900 implements the functions of the third device in the above method embodiments.

[0212] Optionally, the smart device 900 can be an FTTR master device, an FTTR slave device, or a smart device such as a smart camera or a smart door lock; this application does not limit this.

[0213] This application provides a first device, as shown in FIG10, the first device 1000 including: a transmitting unit 1010 and a receiving unit 1020.

[0214] The receiving unit 1020 is configured to receive at least one first message from at least one second device, wherein each first message corresponds to at least one second device, and the first message includes the resource status of the corresponding second device; and to obtain the execution request of the target task.

[0215] The sending unit 1010 is used to send a second message to a target device in at least one second device. The second message is used to instruct the target device to process a target subtask in a target task. The target subtask is based on the resource allocation of the target device.

[0216] The receiving unit 1020 is also used to receive a report message from the target device, the report message including the processing result of the target subtask.

[0217] The first device 1000 is used to implement the computing resource processing method shown in any of the embodiments in Figures 5 to 8.

[0218] This application provides a second device, as shown in FIG11, the second device 1100 including: a transmitting unit 1110 and a receiving unit 1120.

[0219] The sending unit 1110 is used to send a first message to the first device, the first message including the resource status of the second device.

[0220] The receiving unit 1120 is used to receive a second message from the first device. The second message is used to instruct the second device to process the target sub-task in the target task. The target sub-task is based on the resource allocation of the second device.

[0221] The sending unit 1110 is also used to send a reporting message to the first device, the reporting message including the processing result of the target subtask.

[0222] The second device 1100 is used to implement the computing resource processing method shown in any of the embodiments in Figures 5 to 8.

[0223] This application provides a third device, as shown in FIG12, the third device 1200 including: a transmitting unit 1210 and a receiving unit 1220.

[0224] The sending unit 1210 is used to send a third message to the first device, the third message being used to request the processing of the target task.

[0225] The receiving unit 1220 is configured to: receive a fourth message from the first device, the fourth message including the processing result of the target task, the processing result of the target task being the integrated processing results of sub-tasks reported by different second devices, and the sub-tasks processed by different second devices being included in the target task.

[0226] The third device 1200 is used to implement the computing resource processing method shown in any of the embodiments in Figures 5 to 8.

[0227] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a program that, when executed by a computer, performs the method shown in any of the embodiments in Figures 5 to 8.

[0228] The computer-readable storage medium can be a USB flash drive, a portable hard drive, a floppy disk, an optical disc, a memory card, or other types of storage media; this application does not limit the scope of the application.

[0229] This application also provides a computer program product. When the computer program product is executed on a computer, the computer performs the method shown in any of the embodiments in Figures 5 to 8.

[0230] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0231] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0232] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0233] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0234] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data processing network, characterized in that, include: The system includes a computing power demand module, a computing power management module, and at least one computing execution module, wherein the computing power management module and the at least one computing execution module are deployed on different devices. The computing power requirement module is used to publish the execution request of the target task; The computing power management module is used to obtain the resource status of the at least one computing execution device, and allocate the subtasks in the target task to the at least one computing execution module for execution according to the resource status of the at least one computing execution module; The at least one computing execution module is used to process the sub-tasks assigned by the computing power management module and to feed back the processing results of the sub-tasks to the computing power management module. The computing power management module is also used to integrate the processing results of the subtasks into the processing results of the target task, and to feed back the processing results of the target task to the computing power demand module.

2. The network according to claim 1, characterized in that: The computing power demand module and the computing power management module are both deployed on the first device, and the at least one computing execution module is deployed on at least one second device; or... The computing power management module is deployed on the first device, the computing power demand module is deployed on the third device, and the at least one computing execution module is deployed on at least one second device.

3. The network according to claim 1 or 2, characterized in that, The computing power management module is also used for: Send the execution status of the target task to the computing power demand module.

4. The network according to claim 3, characterized in that, The execution state includes at least one of the following: Success, failure, execution progress, and processing results.

5. The network according to claim 3 or 4, characterized in that, The computing power requirement module is also used for: Query the execution status of the target task from the computing power management module.

6. The network according to any one of claims 1 to 5, characterized in that: The computing power management module is further configured to: send at least one configuration information to the at least one computing execution module; The at least one computation execution module is further configured to: perform initialization based on the at least one configuration information.

7. The network according to any one of claims 1 to 6, Its characteristics are: The at least one computing execution module is further configured to: send a keep-alive message to the computing power management module; The computing power management module is also used to: send a keep-alive response message to the at least one computing execution module.

8. The network according to any one of claims 1 to 7, characterized in that; The computing power management module is also used to: query the current resource status from the at least one computing execution module; The at least one computing execution module is further configured to: reply to the computing power management module with the current resource status.

9. The network according to any one of claims 1 to 8, characterized in that; The computing power management module is further configured to: send a deregistration instruction to the target computing execution module in the at least one computing execution module.

10. A data processing method, characterized in that, The method, applied to a first device in a network, the network further including at least one second device, comprises: Receive at least one first message from the at least one second device, wherein the at least one first message corresponds one-to-one with the at least one second device, and the first message includes the resource status of the corresponding second device; Obtain the execution request for the target task; A second message is sent to the target device among the at least one second device, the second message being used to instruct the target device to process a target subtask in the target task, the target subtask being based on the resource allocation of the target device; Receive a report message from the target device, the report message including the processing result of the target subtask.

11. The method according to claim 10, characterized in that, The step of obtaining the execution request for the target task includes: Receive a third message from a third device, the third message being used to request processing of the target task; After receiving the reporting message from the target device, the method further includes: A fourth message is sent to the third device, the fourth message including the processing result of the target task, the processing result of the target task being obtained by integrating the processing results of sub-tasks reported by different second devices.

12. The method according to claim 11, characterized in that, After sending the second message to the target device in the at least one second device, the method further includes: Send a task status to the third device, the task status being used to indicate the execution status of the target task.

13. The method according to claim 12, characterized in that, Before sending the task status to the third device, the method further includes: A query request is received from the third device, the query request being used to query the execution status of the target task.

14. The method according to any one of claims 10 to 13, characterized in that, After receiving at least one first message from the at least one second device and before obtaining the execution request for the target task, the method further includes: At least one configuration message is sent to the at least one second device, the at least one configuration message corresponding to the at least one second device, the configuration message including configuration information of the corresponding second device, the configuration information being used to initialize the second device.

15. The method according to any one of claims 10 to 14, characterized in that, The method further includes: Receive keep-alive message from the at least one second device; Send a keep-alive response message to the at least one second device.

16. The method according to any one of claims 10 to 15, characterized in that, The method further includes: Send a query message to the at least one second device, the query message being used to query the current resource status of the second device; Receive a query response message from the at least one second device, the query response message being used to indicate the current resource status of the second device.

17. The method according to any one of claims 10 to 16, characterized in that, The method further includes: Send a deregistration instruction to the target device in the at least one second device.

18. A data processing method, characterized in that, The method is applied to a second device in a network, the network further comprising the first device, the method comprising: Send a first message to the first device, the first message including the resource status of the second device; Receive a second message from the first device, the second message being used to instruct the second device to process a target subtask in a target task, the target subtask being based on the resource allocation of the second device; A reporting message is sent to the first device, the reporting message including the processing result of the target subtask.

19. The method according to claim 18, characterized in that, After sending the first message to the first device, the method further includes: Receive a configuration message from the first device, the configuration message including configuration information of the second device; The second device is initialized based on the configuration information.

20. The method according to claim 18 or 19, characterized in that, The method further includes: Send a keep-alive message to the first device; Receive a keep-alive response message from the first device.

21. The method according to any one of claims 18 to 20, characterized in that, The method further includes: Receive a query message from the first device, the query message being used to query the current resource status of the second device; A query response message is sent to the first device, the query response message being used to indicate the current resource status of the second device.

22. The method according to any one of claims 18 to 21, characterized in that, The method further includes: Receive a deregistration instruction from the first device.

23. A data processing method, characterized in that, The method, applied to a third device in a network, the network further comprising a first device and at least one second device, includes: Send a third message to the first device, the third message being used to request processing of the target task; A fourth message is received from the first device. The fourth message includes the processing result of the target task. The processing result of the target task is obtained by integrating the processing results of sub-tasks reported by different second devices. The sub-tasks processed by different second devices are all included in the target task.

24. The method according to claim 23, characterized in that, After sending the third message to the first device, the method further includes: Receive a task status from the first device, the task status being used to indicate the execution status of the target task.

25. The method according to claim 24, characterized in that, Before receiving the task status from the first device, the method further includes: A query request is sent to the first device, the query request being used to query the execution status of the target task.

26. A smart device, characterized in that, It includes a processor and an interface circuit, wherein the processor and the interface circuit are coupled to each other; The interface circuit is used to receive signals from other devices besides the smart device and transmit them to the processor, or to send signals from the processor to other devices besides the smart device. The processor is configured to perform the method as described in any one of claims 10 to 17, 18 to 22, or 23 to 25.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed by the computer, performs the method as described in any one of claims 10 to 17, 18 to 22, or 23 to 25.

28. A computer program product, characterized in that, When the computer program product is executed on a computer, the computer performs the method as described in any one of claims 10 to 17, 18 to 22, or 23 to 25.

Images