Multi-modal scheduling method based on early exit under edge limited equipment

Abstract

The invention discloses a multi-modal scheduling method based on early quit under edge limited equipment, and relates to the field of resource unloading. The method comprises the following steps: establishing a multi-modal scheduling framework which comprises an edge device end and a server end; obtaining a plurality of early quit nodes by using an early quit mechanism; the method comprises the following steps: establishing a multi-modal unloading scheduling system model based on Lyapunov optimization according to network environment information and an early exiting node under the maximum delay constraint, and taking Lyapunov drift penalty minimization as an optimization target; generating a modal unloading strategy by using the multi-modal unloading scheduling system model; according to the modal unloading strategy, the edge device side generates a first modal identification result, adjusts the resolution of the second spectral data and then transmits the second spectral data to the server side, and the server side generates a second modal identification result; and carrying out probability integration on the first modal identification result and the second modal identification result at the edge equipment end to obtain a prediction result. Compared with the prior art, the system stability and the model processing accuracy can be ensured, and the processing speed of the system can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of resource unloading, and more specifically, to a multi-modal scheduling method based on early exit in edge-constrained devices. Background Art

[0002] Multimodal collaborative processing is to enable the model to obtain data from multiple modalities and then collaboratively obtain the final result. Compared with traditional single-modal processing, multimodality can improve the accuracy of the model. For example, in multispectral object recognition, the use of infrared data and RGB data improves the success rate of model detection.

[0003] Due to many factors such as cost, most models need to be deployed on edge devices with limited computing power (referred to as "edge-constrained devices"). Although multimodality improves model accuracy, it also increases the amount of model calculation. Therefore, it is particularly important to reasonably schedule multimodality so that edge devices can meet certain accuracy and improve running speed. In the past, many studies have focused on how to offload tasks from edge devices to servers in order to improve the processing speed of edge devices. In the past, dynamic programming methods were often used to find the optimal solution by pruning the entire search space. However, when the model accuracy and the entire environment are changing, the entire scheduling problem becomes an NP-hard problem, and it is difficult to find the optimal solution in a limited time. Therefore, some researchers began to use methods such as ant colony algorithms and simulated annealing to schedule the resource offloading of edge devices, but there is too much uncertainty and the entire system cannot be kept in a stable state. Recently, some people have proposed using reinforcement learning to offload the resources of single-modal models on edge devices. They use reinforcement learning to schedule the resources of edge devices to decide whether to offload, but reinforcement learning also brings additional computing power.

[0004] In addition, previous computing task offloading scheduling algorithms have all used a single mode as the experimental object. In a multimodal model, the combination of different modes will bring different processing results, which is far more complicated than the multimodal situation. Therefore, the scheduling algorithm needs to dynamically schedule the input of each mode according to the environmental conditions, so that it can both ensure the accuracy of the model and improve the system processing speed. Summary of the invention

[0005] In order to overcome the defect of high computational overhead of deploying the multimodal model on edge devices described in the above-mentioned prior art, the present invention provides a multimodal scheduling method based on early exit under edge constrained devices.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] In a first aspect, a multimodal scheduling method based on early exit under edge-constrained devices is applied to the field of multispectral object recognition, including:

[0008] Establish a multimodal scheduling framework, including an edge device end and a server end; wherein the edge device end is deployed with a first modal model for processing first spectral data to obtain a first modal recognition result, and the server end is deployed with a second modal model for processing second spectral data to obtain a second modal recognition result;

[0009] Obtaining a plurality of early exit nodes about the first modal model by using an early exit mechanism;

[0010] Under the maximum delay constraint, according to the network environment information and the early exit node, a multi-modal offloading scheduling system model is established based on Lyapunov optimization, and the minimization of Lyapunov drift penalty is taken as the optimization goal;

[0011] Generate a modal offloading strategy using the multimodal offloading scheduling system model;

[0012] According to the modality offloading strategy, the edge device obtains a first modality recognition result based on the first spectral data, adjusts the resolution of the second spectral data and transmits it to the server, and the server obtains a second modality recognition result based on the second spectral data;

[0013] At the edge device end, the first modal recognition result and the second modal recognition result are probabilistically integrated to obtain a prediction result.

[0014] In a second aspect, an electronic device includes:

[0015] A memory for storing computer executable instructions or computer programs;

[0016] The processor is used to implement the method described in the first aspect when executing the computer executable instructions or computer program stored in the memory.

[0017] In a third aspect, a computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by a processor to implement the method described in the first aspect.

[0018] In a fourth aspect, a computer program product comprises a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the method of the first aspect is implemented.

[0019] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0020] The present invention establishes a multimodal scheduling framework including an edge device end and a server end, and a multimodal unloading scheduling system model based on Lyapunov optimization, uses an early exit mechanism to obtain several early exit nodes about the first modal model, takes minimization of Lyapunov drift penalty as the optimization goal, and uses a multimodal unloading scheduling system model to generate a modal unloading strategy. Then, the edge device end generates a first modal recognition result according to the modal unloading strategy, and adjusts the resolution of the second spectral data before transmitting it to the server end for processing. At the edge device end, the first modal recognition result and the second modal recognition result are probabilistically integrated based on very late fusion to obtain a prediction result. Compared with the prior art, the present invention can reduce the computational overhead of the edge device end, and utilize the characteristics of multimodality in edge devices with limited computing resources, which can not only ensure system stability and model processing accuracy, but also improve the processing speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the multi-modal scheduling method in Example 1 of the present application;

[0022] Figure 2 This is a schematic diagram of a multi-modal scheduling framework in Example 1 of the present application;

[0023] Figure 3 This is a schematic diagram of a thermal-sensitive model with an early exit node in Example 1 of the present application. DETAILED DESCRIPTION

[0024] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged in appropriate cases, which is only to describe the distinction mode adopted by the objects of the same attributes in the embodiments of the present application when describing. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment containing a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment. The term "determine" widely covers various actions, and may include acquisition, calculation, calculation, processing, derivation, investigation, search (for example, search in a table, database or other data structure), ascertainment, and similar actions, and may also include reception (for example, receiving information), access (for example, accessing data in a memory) and similar actions, and may also include generation, creation, establishment and similar actions, as well as parsing, selection, selection and similar actions, etc. The relevant definitions of other terms will be given in the following description.

[0025] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.

[0026] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0027] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0028] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0029] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0030] Example 1

[0031] In this embodiment, an infrared modality image is used as the first spectral data, an RGB modality image is used as the second spectral data, a thermal model is used as the first modality model, and an RGB model is used as the second modality model to provide a multimodal scheduling method based on early exit under edge-constrained devices, which is applied to the field of multispectral object recognition. Figure 1 ,include:

[0032] S10. Establish a multimodal scheduling framework, see Figure 2 , including an edge device end and a server end; wherein the edge device end is deployed with a first modal model for processing first spectral data to obtain a first modal recognition result, and the server end is deployed with a second modal model for processing second spectral data to obtain a second modal recognition result.

[0033] S20. Obtain several early exit nodes about the first modal model using an early exit mechanism.

[0034] S30. Under the maximum delay constraint, according to the network environment information and the early exit node, based on Lyapunov optimization, a multimodal offloading scheduling system model is established, and the minimization of Lyapunov drift penalty is taken as the optimization goal.

[0035] S40: Generate a modal offloading strategy using the multi-modal offloading scheduling system model.

[0036] S50. According to the modal offloading strategy, the edge device obtains a first modal recognition result based on the first spectral data, adjusts the resolution of the second spectral data, and transmits the result to the server, and the server obtains the second modal recognition result based on the second spectral data.

[0037] S60: At the edge device, probabilistically integrate the first modal recognition result and the second modal recognition result to obtain a prediction result.

[0038] Those skilled in the art can understand that, in multimodal collaborative processing, feature fusion is mainly divided into early fusion, mid-term fusion and very late fusion, among which, early fusion is to jointly process the two modal data in the early stage of data processing; mid-term fusion processes the two modalities separately first, and then fuses the intermediate feature results in the middle; very late fusion uses the probabilistic embedding method to fuse the results of the two modal processing. In this embodiment, very late fusion is used as the multimodal scheduling scenario.

[0039] It should be noted that the first modal model is a network model based on the chain rule, that is, the result calculated in the previous stage of the model will be passed to the next stage, and finally an output result is obtained, as shown in the following formula:

[0040] F(x)=f L (f L-1 (…(f1(x)))) (1)

[0041] It should also be noted that the scheduling method needs to dynamically schedule the input of each modality according to the environmental conditions, so that it can both ensure the accuracy of the model and improve the system processing speed. This embodiment considers adjusting the resolution of the RGB image by the edge device and then unloading it to the server for processing. The impact of images of different resolutions on the model accuracy can be regarded as a convex function, as shown in the following formula:

[0042]

[0043] Wherein, A1 indicates the result of the RGB model (i.e., the first modal model), a, b, and d are fitting parameters, and e is the base of the natural logarithm.

[0044] In some specific implementations, the impact of different resolutions on the accuracy of the RGB model is obtained by testing the dataset FLIR_ADAS_v2, and the impact of different resolutions is used as the initial value for Lyapunov optimization.

[0045] It can be understood that this embodiment can reduce the computing overhead on the edge device side, utilize the multimodal characteristics on edge devices with limited computing resources, and schedule the input of the multimodal model according to the current computing power and network status of the device, which can not only ensure the system stability and model processing accuracy but also improve the processing speed of the system.

[0046] In some preferred embodiments, the first model is trained by a preset training set to obtain a plurality of the early exit nodes; wherein the early exit nodes are represented by the average accuracy of the model and are marked with the corresponding first model processing time.

[0047] It can be understood that by using the early exit mechanism to control the training of the first modal model, multiple early exit nodes can be obtained.

[0048] Specifically, by testing the dataset FLIR_ADAS_v2, the average model accuracy r of each exit point is obtained. n and calculation time tt n (i.e., the first model processing time), a heat-sensitive model with early exit nodes such as Figure 3 shown.

[0049] It should be noted that due to network fluctuations, the highest RGB resolution (image frame quality) is always considered. Although offloading can obtain high-precision results, it also brings a large delay as a cost. Therefore, Lyapunov optimization is used to ensure the stability of the system.

[0050] In some optional embodiments, the Lyapunov optimization includes:

[0051] Based on Lyapunov optimization, a virtual queue Q(t) is introduced, and the initial value is set to 0. The expression of the state at time t+1 is as follows:

[0052] Q(t+1)=max{Q(t)+L all -L max ,0} (3)

[0053] L all =max(L rgb ,tt n ) (4)

[0054] L rgb = l tran +s (5)

[0055]

[0056] Where, L max represents the maximum delay constraint; L all Indicates the total time spent by the system; tt nIndicates selection r n The first model processing time required by the first modal model when the early exit node is scheduled at time t; L rgb represents the delay of unloading the second spectral data to the server and completing the processing; s represents the second model processing time on the server side; l tran represents the transmission delay of the second spectral data being unloaded to the server; R represents the size of the second spectral data to be transmitted, B t represents the network bandwidth at time t;

[0057] For each time slot t, define the quadratic Lyapunov function L(t):

[0058]

[0059] The Lyapunov drift penalty is introduced, and the minimization of the Lyapunov drift penalty is used as the optimization goal when generating the modal offloading strategy, so that the multimodal scheduling framework meets the model accuracy and delay in the long-term time T.

[0060] Furthermore, an additional precision term is introduced to adjust the Lyapunov drift penalty. The adjusted Lyapunov drift penalty expression is as follows:

[0061] Δ(Q(t)) ′ =Δ(Q(t))-ωA t (8)

[0062] Where, Δ(Q(t)) ′ represents the Lyapunov drift penalty after adjustment; Δ(Q(t)) represents the Lyapunov drift penalty before adjustment; ωA t represents the additional precision term, ω represents the adjustment parameter, A t Indicates the recognition accuracy of the second spectral data.

[0063] It can be understood that in the actual application process, the system needs to meet the model accuracy and delay under the long-term time T. In the above embodiment, in order to ensure the model accuracy, an additional accuracy item is added to Lyapunov, and ω is used as an adjustment parameter. Every time the system makes a decision, it is necessary to consider minimizing the Lyapunov penalty. Obviously, when the delay is large, the penalty brought by the formula is also greater. When the delay is small, the system can bring better long-term results by improving the accuracy. Through Lyapunov optimization, the system can ensure certain accuracy and delay constraints. When the system has high requirements for model accuracy, the weight of the model accuracy can be increased by increasing the value of ω.

[0064] Furthermore, the generating modal uninstallation strategy includes:

[0065] Obtain the Lyapunov drift penalty at the corresponding time, and determine whether the Lyapunov drift penalty is higher than the preset threshold value Q max :

[0066] If so, the early exit point of the first modal model is selected with a time not exceeding L rgb The most recent early exit point of

[0067] Otherwise, the optimal computational cost-effectiveness is determined according to the overall model accuracy of the multimodal scheduling framework and the total system time spent, and the early exit point and the resolution of the second spectral data unloaded to the server end are determined according to the optimal cost-effectiveness as the modality unloading strategy.

[0068] Furthermore, the expression of the optimal computing cost performance is:

[0069]

[0070] Wherein, P(A1, A2) represents the overall model accuracy; A1 represents the model accuracy of the first modal model under the corresponding modal unloading strategy; A2 represents the model accuracy of the second modal model under the corresponding modal unloading strategy.

[0071] In some specific implementations, the pseudo code of the multi-modal adaptive offloading method on the edge device side is as follows:

[0072] Table 1 Pseudo code of adaptive offloading method

[0073]

[0074] Assuming that x1 represents the output of the RGB model, x2 represents the output of the thermal model, y represents the object label, and p represents the final predicted probability (i.e., the prediction result), the effectiveness of probability integration is as follows:

[0075] p(x1,x2|y)=p(x1|y)p(x2|y) (10)

[0076]

[0077] In some preferred embodiments, the process expression of the probability integration is as follows:

[0078]

[0079] In the formula, x i represents the information of the i-th modality; p(·) represents the prediction result; M represents the total number of modalities; y represents the object label.

[0080] It should be emphasized that this probability-based probability integration method can be extended to more modes and is scalable.

[0081] In some preferred embodiments, in order to facilitate the continuous operation of the multimodal scheduling framework, the virtual queue Q at time slot t+1 is continuously updated. t+1 , repeat steps S30-S60 to achieve adaptive dynamic scheduling.

[0082] Example 2

[0083] This embodiment provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor, so that the processor executes part or all of the steps of the method provided in Example 1 of the present application.

[0084] It is understood that the storage medium may be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0085] Exemplarily, the processor may be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA).

[0086] Exemplarily, the read-only memory includes but is not limited to MASK ROM, PROM, EPROM, EEPROM, Flash, etc.

[0087] Exemplarily, the random access memory includes but is not limited to DRAM, SRAM, SDRAM, DDR SDRAM, etc.

[0088] In some examples, a computer program product is provided, which can be implemented in hardware, software or a combination thereof. As a non-limiting example, the computer program product can be embodied as the storage medium, or as a software product, such as an SDK (Software Development Kit).

[0089] As a non-limiting example, a computer program product is provided, the computer program product includes a computer program or a computer executable instruction, the computer program or the computer executable instruction is stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs some or all steps of the method described in the embodiment of the present application.

[0090] In some examples, a computer program is provided, comprising a computer-readable code, and when the computer-readable code is run in a computer device, a processor in the computer device executes a part or all of the steps in the method.

[0091] This embodiment also proposes an electronic device, including a memory and a processor, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and when the processor executes the at least one instruction, at least one program, a code set or an instruction set, it implements part or all of the steps of the method described in Example 1.

[0092] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory and a communication interface; wherein the processor generally controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers through a network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed or processed by the processor and various modules in the electronic device (including but not limited to image data, audio data, voice communication data and video communication data), which can be implemented by flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or random access memory (RAM).

[0093] The processor may include one or more processing elements. Thus, the processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. In addition, each integrated circuit may include a circuit (e.g., a first circuit, a second circuit, and other circuits, etc.) configured to perform the functions of the processor.

[0094] Furthermore, data may be transmitted between the processor, the communication interface and the memory via a bus, which may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together.

[0095] It can be understood that the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described again here.

[0096] The same or similar reference numerals correspond to the same or similar components;

[0097] The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present application;

[0098] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0099] In different specific implementations, the method or system described in the present application can be implemented in software, hardware or a combination thereof. In addition, the order of the steps of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0100] Obviously, the above-mentioned embodiments of the present application are merely examples for clearly illustrating the present application, and are not intended to limit the implementation methods of the present application, and are not intended to limit the present application. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. Each discrete structure / function module or unit can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part, and the structure and function of the discrete components can be implemented as a combined structure or component. It is not necessary and impossible to enumerate all the implementation methods here. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A multi-modal scheduling method based on early exit under edge-constrained devices, characterized in that: Applications in the field of multispectral object recognition include: Establish a multimodal scheduling framework, including an edge device end and a server end; wherein the edge device end is deployed with a first modal model for processing first spectral data to obtain a first modal recognition result, and the server end is deployed with a second modal model for processing second spectral data to obtain a second modal recognition result; Obtaining a plurality of early exit nodes about the first modal model by using an early exit mechanism; Under the maximum delay constraint, according to the network environment information and the early exit node, a multi-modal offloading scheduling system model is established based on Lyapunov optimization, and the minimization of Lyapunov drift penalty is taken as the optimization goal; Generate a modal offloading strategy using the multimodal offloading scheduling system model; According to the modality offloading strategy, the edge device obtains a first modality recognition result based on the first spectral data, adjusts the resolution of the second spectral data and transmits it to the server, and the server obtains the second modality recognition result based on the second spectral data; At the edge device end, the first modal recognition result and the second modal recognition result are probabilistically integrated to obtain a prediction result.

2. According to claim 1, a multi-modal scheduling method based on early exit for edge-constrained devices is characterized in that: The first model is trained by a preset training set to obtain a plurality of the early exit nodes; wherein the early exit nodes are represented by the average accuracy of the model and are marked with the corresponding first model processing time.

3. The multi-modal scheduling method based on early exit for edge-constrained devices according to claim 2, characterized in that: The Lyapunov optimization includes: Based on Lyapunov optimization, a virtual queue Q(t) is introduced, and the initial value is set to 0. The expression of the state at time t+1 is as follows: Q(t+1)=max{Q(t)+L all -L max ,0} L all =max(L rgb ,tt n ) L rgb =l tran +s Where, L max represents the maximum delay constraint; L all Indicates the total time spent by the system; tt n Indicates selection r n The first model processing time required by the first modal model when the early exit node is scheduled at time t; L rgb represents the delay of unloading the second spectral data to the server and completing the processing; s represents the second model processing time on the server side; l tran represents the transmission delay of the second spectral data being unloaded to the server; R represents the size of the second spectral data to be transmitted, B t represents the network bandwidth at time t; For each time slot t, define the quadratic Lyapunov function L(t): The Lyapunov drift penalty is introduced, and the minimization of the Lyapunov drift penalty is used as the optimization goal when generating the modal offloading strategy, so that the multimodal scheduling framework meets the model accuracy and delay in the long-term time T.

4. The multi-modal scheduling method based on early exit for edge-constrained devices according to claim 3, characterized in that: The precision additional term is introduced to adjust the Lyapunov drift penalty. The expression of the adjusted Lyapunov drift penalty is as follows: Δ(Q(t))′=Δ(Q(t))-ωA t Wherein, Δ(Q(t))′ represents the Lyapunov drift penalty after adjustment; Δ(Q(t)) represents the Lyapunov drift penalty before adjustment; ωA t represents the additional precision term, ω represents the adjustment parameter, A t Indicates the recognition accuracy of the second spectral data.

5. The multi-modal scheduling method based on early exit for edge-constrained devices according to claim 4, characterized in that: The generating modal uninstallation strategy includes: Obtain the Lyapunov drift penalty at the corresponding time, and determine whether the Lyapunov drift penalty is higher than the preset threshold value Q max : If so, the early exit point of the first modal model is selected with a time not exceeding L rgb The most recent early exit point of Otherwise, the optimal computational cost-effectiveness is determined according to the overall model accuracy of the multimodal scheduling framework and the total system time spent, and the early exit point and the resolution of the second spectral data unloaded to the server end are determined according to the optimal cost-effectiveness as the modality unloading strategy.

6. The multi-modal scheduling method based on early exit for edge-constrained devices according to claim 5, characterized in that: The expression of the optimal computational cost performance is: Wherein, P(A1, A2) represents the overall model accuracy; A1 represents the model accuracy of the first modal model under the corresponding modal unloading strategy; A2 represents the model accuracy of the second modal model under the corresponding modal unloading strategy.

7. A multi-modal scheduling method based on early exit for edge-constrained devices according to any one of claims 1 to 6, characterized in that: The process expression of the probability integration is as follows: In the formula, x i represents the information of the i-th modality; p(·) represents the prediction result; M represents the total number of modalities; y represents the object label.

8. An electronic device, characterized in that: include: A memory for storing computer executable instructions or computer programs; A processor, used to implement the method according to any one of claims 1 to 7 when executing computer executable instructions or computer programs stored in the memory.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images