Model-based signal inference method and apparatus

Abstract

This application provides a model-based signal inference method and apparatus. The model-based signal inference method includes: receiving an input signal; performing inference on the input signal according to dependency information in a preset model to obtain an output signal; wherein the preset model includes operators, and each operator includes input parameters and output parameters; the dependency information is used to indicate the dependency relationship between the input parameters or output parameters of at least one operator in the target frame inference and the input parameters or output parameters of at least one operator in the reference frame inference when performing multi-frame inference on the input signal using the preset model, the target frame inference being at least one frame in multi-frame inference, and the reference frame inference being at least one frame inference among other frames in multi-frame inference besides the target frame inference. Based on the dependency information in the preset model, multi-frame inference with dependencies can be executed in parallel, improving the performance of multi-frame inference.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a model-based signal reasoning method and apparatus. Background Technology

[0002] With the increasing prevalence of deep learning applications, various training and inference frameworks have emerged. When a model is trained and deployed to a production environment, inference performance is a crucial metric. Maximizing computing power without altering hardware configurations can significantly enhance a product's competitiveness.

[0003] Currently, once a model is trained, it can be fed input data and related attribute controls through interfaces provided by deep learning frameworks to perform inference and obtain inference results. During inference, different data can be input and the process repeated; for example, for an image detection model, different frames of a video can be input and inference performed consecutively. If computational resources are abundant, parallel execution can be used to improve efficiency.

[0004] However, when there are data dependencies between inferences, such as when the input of the (n+m)th frame inference depends on the output of the nth frame inference, the inference of the (n+m)th frame must wait until the inference of the nth frame is completed before it can be executed, and parallel execution cannot be used to improve execution efficiency. Thus, when computational resources are abundant, the computing power of these resources cannot be fully utilized, reducing inference performance. Summary of the Invention

[0005] This application provides a model-based signal inference method and apparatus that improves inference performance when there are data dependencies between inferences.

[0006] In a first aspect, embodiments of this application provide a model-based signal inference method, including:

[0007] The system receives an input signal, which is a digital signal that can be processed by a computer or processor. Based on dependency information in a preset model, it performs inference on the input signal using the preset model to obtain an output signal. The preset model includes operators, each with input and output parameters. The dependency information indicates the relationship between the input or output parameters of at least one operator in the target frame inference and the input or output parameters of at least one operator in the reference frame inference when performing multi-frame inference using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame in the multi-frame inference excluding the target frame inference.

[0008] The model-based signal inference method provided in the first aspect allows dependency information in the pre-defined model to indicate the dependencies between the input or output parameters of operators in the target frame inference and the input or output parameters of operators in the reference frame inference when performing multi-frame inference using the model. Based on the dependency information in the pre-defined model, multi-frame inference with dependencies can be executed in parallel at the same time, fully utilizing the computational power of computing resources and improving the inference performance of multi-frame inference.

[0009] Optionally, in one possible implementation of the first aspect, before performing multi-frame inference of the input signal based on the dependency information in the preset model, the method may further include: setting dependency information in the description information of the preset model. The description information is used to describe at least one operator included in the preset model, and the input parameters and output parameters of each operator in the at least one operator.

[0010] Optionally, in one possible implementation of the first aspect, setting dependency information in the description information of the preset model includes: setting dependency information in the description information of the preset model by calling the interface of the preset model, wherein the interface of the preset model is used to provide an entry point for modifying information of the preset model.

[0011] Optionally, in one possible implementation of the first aspect, setting dependency information in the description information of a preset model includes: receiving dependency information input by a user, and setting dependency information in the description information of the preset model based on the dependency information input by the user.

[0012] Optionally, in one possible implementation of the first aspect, before setting the dependency information in the description information of the preset model, the method further includes: obtaining the description information. The dependency information is obtained based on at least one operator included in the description information and the input and output parameters of each operator.

[0013] Optionally, in one possible implementation of the first aspect, inferring the input signal using the preset model based on dependency information in the preset model includes: loading the preset model and dependency information; and scheduling the various operators of the preset model in the current frame inference based on the dependency information.

[0014] Optionally, in one possible implementation of the first aspect, scheduling the operators of the preset model in the current frame inference according to dependency information includes: determining the execution order of the operators of the preset model in the current frame inference according to dependency information; adding the operators of the preset model in the current frame inference to multiple execution queues according to the execution order; wherein, there is no dependency between the input parameters of the multiple operators executed at the same time in the multiple execution queues.

[0015] Secondly, embodiments of this application provide a model-based signal inference apparatus, comprising: a processor and a transmission interface. The transmission interface is used to receive an input signal, which is a digital signal that can be processed by a computer or processor. The processor is used to perform inference on the input signal according to dependency information in a preset model to obtain an output signal. The preset model includes operators, each including input parameters and output parameters; the dependency information indicates the dependency relationship between the input parameters or output parameters of at least one operator in the target frame inference and the input parameters or output parameters of at least one operator in the reference frame inference when performing multi-frame inference on the input signal using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame in the multi-frame inference other than the target frame inference.

[0016] Optionally, in one possible implementation of the second aspect, the processor is further configured to: set dependency information in the description information of the preset model; wherein the description information is used to describe at least one operator included in the preset model and the input parameters and output parameters of each operator in the at least one operator.

[0017] Optionally, in one possible implementation of the second aspect, the processor is specifically configured to: set dependency information in the description information of the preset model by calling the interface of the preset model, wherein the interface of the preset model is used to provide an entry point for modifying information of the preset model.

[0018] Optionally, in one possible implementation of the second aspect, the transmission interface is further configured to receive dependency information input by the user, and the processor is specifically configured to set the dependency information in the description information of the preset model based on the dependency information input by the user.

[0019] Optionally, in one possible implementation of the second aspect, the processor is further configured to: obtain description information; and obtain dependency information based on at least one operator included in the description information and the input and output parameters of each operator.

[0020] Optionally, in one possible implementation of the second aspect, the processor is specifically configured to: load a preset model and dependency information, and schedule the various operators of the preset model in the current frame inference according to the dependency information.

[0021] Optionally, in one possible implementation of the second aspect, the processor is specifically configured to: determine the execution order of each operator of the preset model in the current frame inference based on dependency information; and add each operator of the preset model in the current frame inference to multiple execution queues according to the execution order; wherein, there is no dependency between the input parameters of the multiple operators executed at the same time in the multiple execution queues.

[0022] Thirdly, embodiments of this application provide a model-based signal inference apparatus, comprising: a unit for executing each step of the model-based signal inference method provided in the first aspect above.

[0023] Fourthly, embodiments of this application provide a model-based signal inference apparatus, including a processor for connecting to a memory and calling a program stored in the memory to execute the model-based signal inference method provided in the first aspect above. The memory may be located within or outside the apparatus. The processor may include one or more processors.

[0024] Fifthly, embodiments of this application provide a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the model-based signal reasoning method provided in the first aspect above.

[0025] Sixthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer or processor, implement the model-based signal reasoning method provided in the first aspect above.

[0026] In any of the above aspects, optionally, in one possible implementation, the dependency information includes the dependency between the input parameters of the operator in the target frame inference and the output parameters of the operator in the reference frame inference.

[0027] In one possible implementation, among the above aspects, the dependency information includes a first inter-frame interval, a first operator identifier, and a first parameter identifier. The first inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first input parameter of the first operator in the target frame inference. The first operator identifier is used to indicate the operator in the reference frame inference that is dependent on the first operator, and the first parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first input parameter.

[0028] In all of the above aspects, optionally, in one possible implementation, the dependency information includes the dependency between the output parameters of the operator in the target frame inference and the input parameters of the operator in the reference frame inference.

[0029] In one possible implementation, among the above aspects, the dependency information includes a second inter-frame interval, a second operator identifier, and a second parameter identifier. The second inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first output parameter of the second operator in the target frame inference. The second operator identifier is used to indicate the operator in the reference frame inference that is dependent on the second operator, and the second parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first output parameter.

[0030] In all the foregoing aspects, optionally, in one possible implementation, the processable digital signal includes at least one of the following: image data, voice data, and text data. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating the application scenarios applicable to the embodiments of this application;

[0032] Figure 2 A schematic diagram of the structure of the model provided in the embodiments of this application;

[0033] Figure 3 A flowchart for deploying a deep learning model provided in an embodiment of this application;

[0034] Figure 4 A neural network application architecture diagram provided for embodiments of this application;

[0035] Figure 5 This is a schematic diagram of the structure of multi-frame inference of the model provided in the embodiments of this application;

[0036] Figure 6 Another structural schematic diagram of the model provided in the embodiments of this application;

[0037] Figure 7 A schematic diagram illustrating the descriptive information of the model provided in an embodiment of this application;

[0038] Figure 8 A flowchart illustrating a model-based signal reasoning method provided in this application embodiment;

[0039] Figure 9 A flowchart illustrating another model-based signal reasoning method provided in this application embodiment;

[0040] Figure 10 A schematic diagram of the execution queue in multi-frame inference provided in an embodiment of this application;

[0041] Figure 11 A schematic diagram illustrating the description and dependency information of the model provided in the embodiments of this application;

[0042] Figure 12 Another schematic diagram illustrating the description and dependency information of the model provided in the embodiments of this application;

[0043] Figure 13 A schematic diagram of a model-based signal inference device provided in an embodiment of this application;

[0044] Figure 14 Another schematic diagram of the model-based signal inference device provided in the embodiments of this application;

[0045] Figure 15 This is another schematic diagram of a model-based signal inference device provided in an embodiment of this application. Detailed Implementation

[0046] The embodiments of this application are described below with reference to the accompanying drawings.

[0047] The model-based signal inference method provided in this application can be applied to a model-based signal inference device. When the model-based signal inference device performs continuous multi-frame inference using a preset model, it can execute the model-based signal inference method provided in this application. This application does not limit the specific implementation of the model-based signal inference device; any device with computing power and capable of inference using a model can be used. For example, Figure 1 This is a schematic diagram illustrating an application scenario applicable to the embodiments of this application. For example... Figure 1 As shown, the model-based signal inference device may include a terminal device 11, a server 12, and a computer 13. Optionally, the terminal device 11, server 12, and computer 13 may be connected to a network to acquire or publish data.

[0048] Optionally, terminal device 11 can also be called user equipment (UE), mobile station (MS), or mobile terminal (MT), etc., and is a device that provides voice / data connectivity to users. Examples include handheld devices with wireless connectivity or in-vehicle devices. Currently, some examples of terminal devices include: mobile phones, tablets, laptops, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, and wireless terminals in smart homes, etc.

[0049] It should be noted that, Figure 1 This is merely an example and does not limit the implementation of model-based signal inference devices.

[0050] The concepts involved in the embodiments of this application will be explained below.

[0051] (1) Model

[0052] In fields such as artificial intelligence, machine learning, deep learning, and neural networks, models can be used to process data and achieve specific functions. Using a model, we can perform predictive analysis on the input X and output the result Y. Depending on the specific needs, the model can perform different functions, and the specific forms of X and Y can also differ. Models can be reused. Model reuse can be achieved through parameter settings. For example, for the same scenario and different objects, model reuse can be achieved by assigning different parameter values.

[0053] The model will be illustrated with an example below.

[0054] For example, in the field of image processing, autonomous vehicles can capture images of the road ahead. Inputting this image into a pre-defined model can then output obstacles in the road, achieving obstacle detection. Obstacles can be other vehicles, bicycles, or pedestrians in front of the vehicle.

[0055] For example, in the field of speech processing, a speaker's speech signal can be interfered with by ambient noise. By inputting the mixed signal of the speaker's speech signal and the ambient noise into a preset model, the speaker's speech signal can be output while filtering out the ambient noise, thus achieving signal noise reduction.

[0056] (2) Operators in the model, input parameters of operators, and output parameters of operators.

[0057] In the model, the algorithm is not implemented as a whole. Typically, the algorithm is broken down into multiple, single-granularity operators, and the model is built according to the algorithm's execution logic. Each operator has input and output parameters. For an operator, its input parameters may be the model's input or the output parameters of other operators. Similarly, its output parameters may be the model's output or the input parameters of other operators.

[0058] It should be noted that the embodiments of this application do not limit the number of operators included in the model, the number of input parameters for each operator, and the number of output parameters.

[0059] The following example illustrates this.

[0060] Figure 2 This is a schematic diagram of a model provided in an embodiment of this application. For example... Figure 2 As shown, the model includes four operators, namely operators A to D.

[0061] Operator A has one input parameter x and one output parameter a. The input parameter of operator A is also the input x of the model.

[0062] Operator B has one input parameter a and one output parameter b. The input parameter of operator B is the output parameter a of operator A. The output parameter of operator B is the input parameter b of operator D.

[0063] Operator C has one input parameter a and one output parameter c. The input parameter of operator C is the output parameter a of operator A. The output parameter of operator C is the input parameter c of operator D.

[0064] Operator D has two input parameters, b and c, and one output parameter, y. The input parameter b of operator D is the output parameter b of operator B, and the input parameter c of operator D is the output parameter c of operator C. The output parameter of operator D is the model's output y.

[0065] (3) Model training, inference and deployment

[0066] The model needs to be trained before it can be used for inference and deployment.

[0067] Model training refers to continuously optimizing the parameters of an initial model until it can perform the corresponding functions. There are many ways to train a model, and this application does not limit this method. For example, a model can be trained using a training dataset. For instance, a training dataset includes labeled data. For example, when training a model that implements obstacle detection, the training dataset may include multiple images, each of which has obstacles pre-labeled.

[0068] Model inference refers to the process of inputting data that needs to be processed into a pre-trained model to obtain the output result. For example, let's take an obstacle detection model as an example again. Taking an image as an example, inputting that image into the model will output obstacle information.

[0069] Model deployment refers to the process of applying a trained model to a hardware platform to run the model.

[0070] The following section uses a deep learning model as an example to illustrate the deployment of the model. Figure 3 A flowchart illustrating the deployment of a deep learning model provided in an embodiment of this application. Figure 3 As shown, the deployment model may include:

[0071] S301, Load graph.

[0072] Pre-defined models can be loaded into device memory using the interfaces provided by deep learning frameworks.

[0073] S302, Execute model inference (forward).

[0074] The deep learning framework provides interfaces to load a pre-defined model, providing input data and related attribute controls to perform inference (forward) and obtain the inference result. During the model's inference process, operators are encapsulated within the model; users generally only need to focus on the model's input and output, without needing to concern themselves with the input and output of each individual operator.

[0075] For example, Figure 4 This is a diagram illustrating a neural network application architecture provided in an embodiment of this application. Figure 4 As shown, the neural network application architecture provided in this application embodiment may include: an application entry point 41, a model external interface 42, a deep learning structure 43, a device driver 44, a central processing unit 45, a graphics processing unit 46, a network processor 47, and a digital processor 48. The application entry point 41 is used to select a neural network model. The model external interface 42 is used to call the selected neural network model. The deep learning structure 43 is used to process the input first user image through the neural network model. For example, the deep learning structure 43 may include an environment manager 431, a model manager 432, a task scheduler 433, a task executor 434, and an event manager 435. The environment manager 431 is used to control the startup and shutdown of the device-related environment. The model manager 432 is responsible for operations such as loading and unloading the neural network model. The task scheduler 433 is responsible for managing the sequence in which the neural network models are scheduled. The task executor 434 is responsible for executing tasks of the neural network model. The event manager 435 is responsible for notifications of various events.

[0076] It should be noted that, Figure 4 This is merely an example, and the embodiments in this application do not limit the application architecture of neural networks.

[0077] (4) Inference within each frame and inference across multiple frames

[0078] In model inference, different data can be input and inference can be performed repeatedly. For example, for an image detection model, different frames of a video can be input and inference can be performed continuously. Specifically, for a single frame in the video, the model performs one-frame inference. For multiple frames of the video, the model performs multi-frame inference. In the embodiments of this application, multi-frame inference is also referred to as multiple inferences, continuous inference, multi-frame continuous inference, continuous multi-frame inference, or continuous multiple inferences. Each frame inference is also referred to as each inference or frame inference.

[0079] When performing multi-frame inference, the model can perform inference in a serial manner or in a parallel manner, depending on the model interface type and the availability of computing resources.

[0080] The following is combined with Figure 5 Please provide an explanation. Figure 5 This is a schematic diagram illustrating the structure of a multi-frame inference model provided in an embodiment of this application. It is assumed that the input to each frame of the model's inference does not depend on the output of other frames. For example... Figure 5 As shown, in serial multi-frame inference, the first frame of inference is executed at time T0, the second frame at time T1, and the third frame at time T2. In parallel multi-frame inference, since the input of each frame of the model does not depend on the output of other frames, all three frames can be executed simultaneously at time T0. Therefore, if computational resources are abundant and the input data for each frame of inference is independent, parallel execution of multi-frame inference can improve resource utilization and increase inference speed.

[0081] The following explains the case where the input of each frame of the model's inference depends on the output of the inference in other frames. Figure 6 This is another structural schematic diagram of the model provided in an embodiment of this application. For example... Figure 6 As shown, the model includes operators A to D. Operators A and D are related to... Figure 2 Operators A and D in the model shown are similar and will not be described further here. Operator B has two input parameters. In the second frame of inference, one input parameter of operator B is the output parameter c of operator C in the first frame of inference; similarly, in the third frame of inference, one input parameter of operator B is the output parameter c of operator C in the second frame of inference, and so on. The input of each frame of inference depends on the output of the previous frame of inference. Therefore, if a parallel execution method is used for multi-frame inference, the dependencies between inferences in different frames need to be considered.

[0082] (5) Model description information

[0083] The model's descriptive information describes at least one operator included in the model, as well as the input and output parameters of each operator. This application does not limit the storage method or specific content of the descriptive information within the model. For example, the model's descriptive information can be stored in a file, such as a binary file, a text file, etc.

[0084] The following example illustrates the descriptive information.

[0085] Figure 7 This is a schematic diagram illustrating the descriptive information of the model provided in an embodiment of this application. Figure 7 The description information shown is Figure 6 The model shown is described. For example... Figure 6 and Figure 7 As shown, the model has two input parameters, x and y. The description information indicates that the model has four operators, A through D. Operator A has one input parameter x and one output parameter a. Operator B has two input parameters a and y, and one output parameter b. Operator C has one input parameter a and one output parameter c. Operator D has two input parameters b and c, and one output parameter d. The output parameter c of operator C and the output parameter d of operator D are marked as true, representing the model's output. Figure 7 The description provided does not yet reflect the dependencies between the model's multi-frame inference.

[0086] It should be noted that, Figure 7 This is merely an example and does not limit how the descriptive information can be implemented.

[0087] (6) Model dependency information

[0088] The model's dependency information is used to indicate the dependency relationship between the input or output parameters of at least one operator in the target frame inference and the input or output parameters of at least one operator in the reference frame inference when performing multi-frame inference using a preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame in the multi-frame inference other than the target frame inference. The following description, through various embodiments of this application, will provide a detailed explanation.

[0089] Currently, multi-frame inference is performed using parallel execution, with the constraint that the input of each frame's inference does not depend on the output of other frames' inference. For example... Figure 2 or Figure 5 The scenario shown. However, when there are data dependencies between multi-frame inference (e.g., Figure 6 In the scenario shown, if the input for the (n+m)th frame inference depends on the output for the nth frame inference, then the (n+m)th frame inference cannot be executed until the nth frame inference has finished. Even if the input for the (n+m)th frame inference depends on the output of a single operator in the nth frame inference, it still requires all operators in the nth frame inference to be executed. Thus, when computational resources are abundant, this reduces the computational power of those resources and lowers inference performance.

[0090] To address the aforementioned technical problems, this application provides a model-based signal inference method that can be applied to multi-frame inference. For scenarios where input and output parameters are dependent across different frames, the dependency information of the model can reveal the dependencies between different operators in different frames, enabling parallel execution on an operator-by-operator basis. This fully utilizes the computational power of computing resources and improves inference performance.

[0091] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0092] Figure 8 A flowchart illustrating a model-based signal inference method provided in an embodiment of this application. Figure 8 As shown, the model-based signal inference method provided in this embodiment includes:

[0093] S801 receives input signals, which are digital signals that can be processed by a computer or processor.

[0094] The input signal is the signal to be processed from the preset model, which is a digital signal that can be processed by a computer or processor.

[0095] It should be noted that the functions implemented by the preset model are different, the application scenarios are different, and the specific implementation forms and contents of the input signals and processable digital signals may be different. This embodiment does not limit this.

[0096] Optionally, the input signal may include, but is not limited to, at least one of the following: voice signal, video signal, image signal, text signal, temperature signal, or pressure signal.

[0097] For example, when applied to image processing, object detection, and face recognition scenarios, the input signal can be an image signal. Image signals can be landscape signals captured by a camera, images of a community environment captured by surveillance equipment, facial signals of people obtained by an access control system, facial signals of attendees obtained by a conference system, and so on.

[0098] When applied to audio processing, speech recognition, and voiceprint recognition scenarios, the input signal can be a voice signal. This voice signal can be a voice signal recorded by a recording device, a voice signal received by a mobile phone or landline during a call, a voice signal transmitted by a radio station received by a radio, or a voice signal received by participants in a conference system, etc.

[0099] When applied to language translation and semantic recognition scenarios, the input signal can be a text signal. This text signal can be a TXT text signal, a Word text signal, or a PDF text signal, etc.

[0100] Optionally, the processable digital signal may include, but is not limited to, at least one of the following: voice data, image data, video data, text data, temperature data, or pressure data.

[0101] S802. Based on the dependency information in the preset model, infer the input signal using the preset model to obtain the output signal. The preset model includes operators, and each operator includes input parameters and output parameters. The dependency information indicates the dependency relationship between the input or output parameters of at least one operator in the target frame inference and the input or output parameters of at least one operator in the reference frame inference when performing multi-frame inference using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame inference among the other frames in the multi-frame inference besides the target frame inference.

[0102] Specifically, when performing continuous multi-frame inference using a pre-defined model, dependencies exist between different frames. The dependency information in the pre-defined model indicates these dependencies. Specifically, these dependencies are the relationships between the input or output parameters of operators in the target frame inference and the input or output parameters of operators in the reference frame inference. Thus, based on the dependency information in the pre-defined model, multi-frame inference with dependencies can be executed in parallel simultaneously, fully utilizing computing resources and improving inference performance. Furthermore, in multi-frame inference, operators can be used as execution units. Operators without dependencies can be executed in parallel simultaneously, while operators with dependencies can be executed sequentially at different times. Therefore, when performing multi-frame inference on a pre-defined model, some operators can be executed in parallel, while others can be executed sequentially, fully utilizing computing resources and improving inference performance.

[0103] The following example illustrates the dependency information of the preset model.

[0104] In one example, please refer to Figure 6 When the model performs multi-frame inference, the inference of two consecutive frames is dependent. The dependency information of the pre-defined model can indicate that the input parameter (c) of operator B in the target frame inference is dependent on the output parameter (c) of operator C in the reference frame inference. Here, the target frame inference can be the inference of the next frame in two consecutive frames, and the reference frame inference can be the inference of the previous frame in two consecutive frames.

[0105] In another example, the dependency information of the pre-defined model can indicate that the input parameters of operator B in the target frame inference are dependent on the output parameters of operator C in the reference frame inference. Here, the target frame inference can be the Nth frame inference and the (N+1)th frame inference, and the reference frame inference can be the (N-2)th frame inference. N is an odd number greater than 2. It can be seen that the input parameters of operator B in the 3rd and 4th frame inferences depend on the output parameters of operator C in the 1st frame inference. Similarly, the input parameters of operator B in the 5th and 6th frame inferences depend on the output parameters of operator C in the 3rd frame inference, and so on.

[0106] In another example, the dependency information of the pre-defined model can indicate that the input parameters of operator B in the target frame inference are dependent on the output parameters of operator C in the first reference frame inference, and the input parameters of operator A in the target frame inference are dependent on the output parameters of operator D in the second reference frame inference. Here, the target frame inference can be the Nth frame inference, the first reference frame inference can be the (N-2)th frame inference, and the second reference frame inference can be the (N-1)th frame inference. N is an integer greater than 2. It can be seen that the input parameters of operator B in the 3rd frame inference depend on the output parameters of operator C in the 2nd frame inference, and the input parameters of operator A in the 3rd frame inference depend on the output parameters of operator D in the 1st frame inference. Similarly, the input parameters of operator B in the 4th frame inference depend on the output parameters of operator C in the 3rd frame inference, and the input parameters of operator A in the 4th frame inference depend on the output parameters of operator D in the 2nd frame inference, and so on.

[0107] It should be noted that this embodiment does not limit the storage method or specific content of the dependency information in the preset model. For example, the dependency information in the preset model can be stored in a file, which can be a binary file, a text file, etc. Optionally, the file and the file corresponding to the description information of the preset model may be the same file or different files. The description information of the preset model can be referred to the explanation of the aforementioned concepts, and the principle is similar, so it will not be repeated here.

[0108] It should be noted that this embodiment does not limit the preset model and can be any model that performs multi-frame inference.

[0109] This embodiment provides a model-based signal inference method, including: receiving an input signal, where the input signal is a digital signal that can be processed by a computer or processor; performing inference on the input signal according to dependency information in a preset model; and obtaining an output signal. The model-based signal inference method provided in this embodiment, based on dependency information in the preset model, enables multi-frame inference with dependencies to be executed in parallel at the same time, fully utilizing the computing power of computing resources and improving the inference performance of multi-frame inference.

[0110] Figure 9 This is a flowchart illustrating another model-based signal inference method provided in an embodiment of this application. This embodiment provides an exemplary implementation of step S802. Figure 9 As shown, S802, performing inference on the input signal based on the dependency information in the preset model, may include:

[0111] S901, Load the preset model and dependency information.

[0112] S902. Schedule each operator of the preset model in the current frame inference according to the dependency information.

[0113] Specifically, during model inference, a preset model and dependency information are loaded. Based on the dependency information, it can be determined whether each operator in the current frame's inference has a dependency relationship with operators in previous inference frames. Therefore, the operators of the preset model in the current frame's inference can be scheduled to execute in parallel. Because this fully utilizes the computing power of resources, the inference performance of multi-frame inference is improved.

[0114] Optionally, in S902, scheduling the operators of the preset model in the current frame inference according to the dependency information may include:

[0115] The execution order of each operator in the preset model during inference in the current frame is determined based on dependency information.

[0116] Based on the execution order, each operator of the preset model in the current frame inference is added to multiple execution queues. The input parameters of the multiple operators executed simultaneously in the multiple execution queues are independent of each other.

[0117] Specifically, the model-based signal inference device may include multiple processors, and this embodiment does not limit the type or number of processors. For example, the processors may include, but are not limited to, at least one of the following: a central processing unit (CPU), a graphics processing unit (GPU), and a network processing unit (NPU). Each processor may correspond to at least one execution queue. This embodiment does not limit the number of execution queues corresponding to a processor.

[0118] Below, with Figure 6 Taking the model shown as an example, combined with Figure 10 This document provides an illustrative example of multi-frame inference for a pre-defined model. Assume there are four execution queues, and each operator takes the same execution time, T. Performing inference on the model for three consecutive frames creates all the operator tasks for those three frames. Assume the first frame of inference has no inter-frame dependencies. The operators in the first frame are labeled 1A, 1B, 1C, and 1D. The operators in the second frame are labeled 2A, 2B, 2C, and 2D. The operators in the third frame are labeled 3A, 3B, 3C, and 3D. The dependency information in the pre-defined model indicates that the input parameter (c) of operator B in the target frame inference is dependent on the output parameter (c) of operator C in the reference frame inference. The target frame inference can be the subsequent frame of two consecutive frames, and the reference frame inference can be the preceding frame of two consecutive frames.

[0119] Specifically, the execution order of operators in three consecutive frames of inference can be determined based on the dependency information in the preset model. For the second frame of inference, operator 2B depends on operator 1C in the first frame and operator 2A in the second frame. Therefore, the execution order of operator 2B must follow operator 1C in the first frame and operator 2A in the second frame. Similarly, for the third frame of inference, operator 3B depends on operator 2C in the second frame and operator 3A in the third frame. The execution order of operator 3B must follow operator 2C in the second frame and operator 3A in the third frame.

[0120] It should be noted that, for each frame of inference, the dependency relationship between the input or output parameters of at least one operator and the input or output parameters of other operators besides the at least one operator can be obtained through the descriptive information in the model. Figure 6 In the model shown, for the first frame of inference, operators 1B and 1C depend on operator 1A, and operator 1D depends on operators 1B and 1C. Therefore, the execution order of operators in the first frame of inference is: operator 1A, operator 1B, operator 1C, operator 1D. Similarly, for the second frame of inference, the execution order of operators is: operator 2A, operator 2B, operator 2C, operator 2D. For the third frame of inference, the execution order of operators is: operator 3A, operator 3B, operator 3C, operator 3D.

[0121] In summary, the operators are added to multiple execution queues according to their execution order, as follows:

[0122] At time T (marked as 1T): Operator A can be started and executed in all 3 frames of inference. Specifically, operator 1A is in execution queue 1, operator 2A is in execution queue 2, and operator 3A is in execution queue 3.

[0123] At time T (labeled 2T): In frame 1 inference, operators 1B and 1C decoupled from operator 1A and began execution. Operator 1B is in execution queue 1, and operator 1C is in execution queue 2. In frame 2 inference, operator 2C decoupled from operator 2A and began execution. Operator 2C is in execution queue 3. In frame 3 inference, operator 3C decoupled from operator 3A and began execution. Operator 3C is in execution queue 4.

[0124] At time T (labeled 3T): In the first frame of inference, operator 1D decouples from operators 1B and 1C, and begins execution. Operator 1D is in execution queue 2. In the second frame of inference, operator 2B decouples from operator 1C, and begins execution. Operator 2B is in execution queue 3. In the third frame of inference, operator 3B decouples from operator 2C, and begins execution. Operator 3B is in execution queue 4.

[0125] At time T (marked as 4T): During the inference of frame 2, operator 2D decouples from operators 2B and 2C, and begins execution. Operator 2D is in execution queue 3. During the inference of frame 3, operator 3D decouples from operators 3B and 3C, and begins execution. Operator 3D is in execution queue 4.

[0126] It is evident that operators in three consecutive frames of inference can be scheduled in parallel, thereby making full use of computing resources and improving the inference performance of multi-frame inference.

[0127] It should be noted that the example above, in which the operator is added to which execution queue, is only one example, and this embodiment does not limit this. For example, at the third time T (3T), operator 1D can also be in execution queue 1, and at the fourth time T (4T), operator 2D can also be in execution queue 1, and operator 3D can also be executed in execution queue 2.

[0128] The model-based signal inference method provided in this embodiment can schedule each operator of the preset model in the current frame inference according to the dependency information by loading a preset model and dependency information, so that the operators can be executed in parallel, making full use of the computing power of computing resources and improving the inference performance of multi-frame inference.

[0129] Based on any of the above embodiments, optionally, before performing multi-frame inference of the input signal according to the dependency information in the preset model (S802), the model-based signal inference method provided in this embodiment may further include:

[0130] Dependency information is set in the description information of the preset model. The description information is used to describe at least one operator included in the preset model, as well as the input parameters and output parameters of each operator in the at least one operator.

[0131] Specifically, the descriptive information of the preset model is used to describe the basic information of the operators in the preset model, including the number of operators, their names, input parameters, and output parameters. However, the descriptive information cannot reflect the dependencies between different frames when the preset model performs multi-frame inference. For example, refer to... Figure 6 and Figure 7 . Figure 7 The description shown indicates that operator B has two input parameters, a and y. However, the dependency between the input parameter y of operator B and operator C cannot be determined from the description. And as... Figure 6It can be seen that for two adjacent frames of inference, the input parameter y of operator B in the later frame of inference is the output parameter c of operator C in the previous frame of inference. By setting dependency information in the description information of the preset model, dependency information can be added to the preset model, thereby enabling parallel execution of multi-frame inference with dependencies based on this dependency information, thus improving the inference performance of multi-frame inference.

[0132] Optionally, in one implementation, dependency information can be set in the description information of the preset model, which may include:

[0133] By calling the interface of the preset model, dependency information can be set in the description information of the preset model. The interface of the preset model is used to provide an entry point for modifying the information of the preset model.

[0134] Optionally, the interface of the preset model may include at least one of the following: a function interface, an application programming interface (API) interface, and an executable command.

[0135] The following example illustrates this.

[0136] In the first example, an executable command can be invoked to set dependency information by adding independent parameters (file or string) to the description information of the preset model. For example, the execution command is:

[0137] Modeltrans orgModel:xxx desModel:yyy frameDependDecs:"{OP(B)input(c)}depend{frame(-1)OP(C)output(c)}"

[0138] In the second example, for different open-source frameworks, such as TensorFlow, function interfaces can be called to set dependency information in the description information of the preset model. For example, the Python interface is provided as follows:

[0139] import xxx

[0140] xxx.add_frame_depend(orgGraph,frameDependInfo,destGraph)

[0141] In the third example, the description information for the preset model can be a binary file or a TXT file. (Reference) Figure 11 . Figure 11 This is a schematic diagram illustrating the description and dependency information of the model provided in an embodiment of this application. Figure 11The left side of the middle section contains the original description information of the preset model, which can be found in [reference]. Figure 7 The relevant descriptions are not repeated here. Executable commands can be invoked to set dependency information in the original description information of the preset model, such as... Figure 11 As shown on the right. The dependency information may include an inter-frame interval of -1, indicating a dependency between the input parameter (c) of operator B in the target frame inference and the output parameter (c) of operator C in the reference frame inference. The target frame inference can be the subsequent frame inference of two consecutive frames, and the reference frame inference can be the preceding frame inference of two consecutive frames. For example, the command to execute is:

[0142] Modeltrans orgModel:{xxx,graphDescFile}desModel:yyy

[0143] Optionally, before setting the dependency information in the description information of the preset model, it may also include:

[0144] Get description information.

[0145] Dependency information is obtained based on at least one operator included in the description information, as well as the input and output parameters of each operator.

[0146] In this implementation method, the dependency information can be obtained by acquiring the description information of the preset model and the basic information of the operators included in the description information, which is a simple implementation method.

[0147] Optionally, in another implementation, dependency information can be set in the description information of the preset model, which may include:

[0148] Receive dependency information input by the user.

[0149] The dependency information is set in the description information of the preset model based on the dependency information input by the user.

[0150] In this implementation, the dependency information can be set directly in the description information of the preset model based on the dependency information input by the user, which is a simple implementation method.

[0151] Below, based on any of the above embodiments, the dependency information in the preset model will be described in detail.

[0152] Optionally, in one implementation, the dependency information may include the dependency between the input parameters of the operator in the target frame inference and the output parameters of the operator in the reference frame inference.

[0153] In this implementation, the target frame inference follows the reference frame inference in time. The input parameters of the operators in the target frame inference depend on the output parameters of the operators in the reference frame inference. For example, ... Figure 6 As shown, the target frame inference can be the second frame inference, and the reference frame inference can be the first frame inference. The input parameters of operator B in the second frame inference are dependent on the output parameters of operator C in the first frame inference.

[0154] Optionally, the dependency information may include a first inter-frame interval, a first operator identifier, and a first parameter identifier. The first inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which are dependent on the first input parameter of the first operator in the target frame inference. The first operator identifier is used to indicate the operator in the reference frame inference that is dependent on the first operator, and the first parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first input parameter.

[0155] The following is combined with Figure 6 and Figure 11 This provides an explanation of the dependency information.

[0156] like Figure 6 and Figure 11 As shown, the target frame inference can be the second frame inference, and the reference frame inference can be the first frame inference. The first operator in the target frame inference is operator B. The first input parameter of the first operator in the target frame inference is parameter c. The first inter-frame interval is the inter-frame interval between the second frame inference and the first frame inference, specifically -1. The first operator is identified as operator C in the first frame inference. The first parameter is identified as the output parameter c of operator C in the first frame inference.

[0157] It should be noted that in this example, the fetching of the first inter-frame interval, the first operator identifier, and the first parameter identifier is just an example. When the preset model is different, the specific fetching of the parameters will be different. This embodiment does not limit the specific fetching.

[0158] Optionally, in another implementation, the dependency information includes the dependency between the output parameters of the operator in the target frame inference and the input parameters of the operator in the reference frame inference.

[0159] In this implementation, the target frame inference precedes the reference frame inference in time. The input parameters of the operators in the reference frame inference depend on the output parameters of the operators in the target frame inference. For example, ... Figure 6 As shown, the target frame inference can be frame 1 inference, and the reference frame inference can be frame 2 inference. The input parameters of operator B in frame 2 inference are dependent on the output parameters of operator C in frame 1 inference.

[0160] Optionally, the dependency information may include a second inter-frame interval, a second operator identifier, and a second parameter identifier. The second inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which are dependent on the first output parameter of the second operator in the target frame inference. The second operator identifier is used to indicate the operator in the reference frame inference that is dependent on the second operator, and the second parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first output parameter.

[0161] The following is combined with Figure 6 and Figure 12 This provides an explanation of the dependency information.

[0162] like Figure 6 and Figure 12 As shown, the target frame inference can be frame 1 inference, and the reference frame inference can be frame 2 inference. The second operator in the target frame inference is operator C. The first output parameter of the second operator in the target frame inference is parameter c. The second inter-frame interval is the inter-frame interval between frame 2 inference and frame 1 inference, specifically 1. The second operator is identified as operator B in frame 2 inference. The second parameter is identified as the input parameter c of operator B in frame 2 inference.

[0163] It should be noted that in this example, the indices for the second inter-frame interval, the second operator identifier, and the second parameter identifier are just examples. When the preset model is different, the specific indices for the parameters will be different. This embodiment does not limit the specific indices.

[0164] Figure 13 This is a schematic diagram of a model-based signal inference device provided in an embodiment of this application. The model-based signal inference device provided in this embodiment can execute the model-based signal inference method provided in the embodiment of this application. Figure 13 As shown, the model-based signal inference device provided in this embodiment may include:

[0165] Receiver module 21 is used to receive input signals, wherein the input signals are digital signals that can be processed by a computer or processor;

[0166] Processing module 22 is used to perform inference on the input signal based on the dependency information in the preset model to obtain an output signal;

[0167] The preset model includes operators, each including input parameters and output parameters. The dependency information is used to indicate the dependency relationship between the input parameters or output parameters of at least one operator in the target frame inference and the input parameters or output parameters of at least one operator in the reference frame inference when performing multi-frame inference on the input signal using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame inference among the other frames in the multi-frame inference besides the target frame inference.

[0168] Optionally, the processing module 22 is further configured to:

[0169] The dependency information is set in the description information of the preset model; wherein the description information is used to describe at least one operator included in the preset model and the input parameters and output parameters of each operator in the at least one operator.

[0170] Optionally, the processing module 22 is specifically used for:

[0171] The dependency information is set in the description information of the preset model by calling the interface of the preset model. The interface of the preset model is used to provide an entry point for modifying information of the preset model.

[0172] Optionally, the receiving module 21 is further configured to receive the dependency information input by the user;

[0173] The processing module 22 is specifically used to set the dependency information in the description information of the preset model according to the dependency information input by the user.

[0174] Optionally, the processing module 22 is further configured to:

[0175] Obtain the description information;

[0176] The dependency information is obtained based on at least one operator included in the description information, as well as the input and output parameters of each operator.

[0177] Optionally, the dependency information includes the dependency relationship between the input parameters of the operator in the target frame inference and the output parameters of the operator in the reference frame inference.

[0178] Optionally, the dependency information includes a first inter-frame interval, a first operator identifier, and a first parameter identifier. The first inter-frame interval is used to indicate the inter-frame interval between the reference frame inference, which is dependent on the first input parameter of the first operator in the target frame inference, and the target frame inference. The first operator identifier is used to indicate the operator in the reference frame inference that is dependent on the first operator, and the first parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first input parameter.

[0179] Optionally, the dependency information includes the dependency relationship between the output parameters of the operator in the target frame inference and the input parameters of the operator in the reference frame inference.

[0180] Optionally, the dependency information includes a second inter-frame interval, a second operator identifier, and a second parameter identifier. The second inter-frame interval is used to indicate the inter-frame interval between the reference frame inference, which is dependent on the first output parameter of the second operator in the target frame inference, and the target frame inference. The second operator identifier is used to indicate the operator in the reference frame inference that is dependent on the second operator, and the second parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first output parameter.

[0181] Optionally, the processing module 22 is specifically used for:

[0182] Load the preset model and the dependency information;

[0183] The operators of the preset model in the current frame inference are scheduled according to the dependency information.

[0184] Optionally, the processing module 22 is specifically used for:

[0185] The execution order of each operator of the preset model in the current frame inference is determined based on the dependency information;

[0186] According to the execution order, each operator of the preset model in the current frame inference is added to multiple execution queues; wherein, there is no dependency between the input parameters of the multiple operators executed at the same time in the multiple execution queues.

[0187] Optionally, the processable digital signal includes at least one of the following: image data, voice data, and text data.

[0188] The model-based signal inference device provided in this embodiment can execute the model-based signal inference method provided in this application embodiment. The technical principles and technical effects are similar, and will not be described again here.

[0189] It should be understood that the module division in the above device is merely a logical functional division. In actual implementation, all or part of the modules can be integrated into a single physical entity, or they can be physically separated. Furthermore, all modules in the device can be implemented entirely through software calls from processing elements, or entirely through hardware, or some modules can be implemented through software calls from processing elements while others are implemented in hardware. For example, each module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as a program in memory, called and executed by a processing element of the device. Moreover, these modules can be integrated together or implemented independently. The processing element mentioned here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be implemented through integrated logic circuits in the processor element or through software calls from processing elements.

[0190] In one example, the modules in the above apparatus can be one or more integrated circuits configured to implement the above methods, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. As another example, when the modules in the above apparatus can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a CPU or other processor capable of calling programs. Furthermore, these modules can be integrated together to implement a system-on-a-chip (SOC).

[0191] In one example, the module used for receiving signals may be the interface circuit of the above device, used to transmit signals with other devices.

[0192] Figure 14 This is another schematic diagram of a model-based signal inference device provided in an embodiment of this application. The model-based signal inference device provided in this embodiment can execute the model-based signal inference method provided in the embodiment of this application. Figure 14As shown, the model-based signal inference apparatus provided in this embodiment may include a processor 31 and a transmission interface 32. The transmission interface 32 can communicate with other devices, and the processor 31 can execute the model-based signal inference method provided in this embodiment. Optionally, the model-based signal inference apparatus may further include a memory.

[0193] It should be noted that this embodiment does not limit the type of model-based signal inference device. For example, a model-based signal inference device can be... Figure 1 The terminal devices, servers, or computers shown are examples of such devices.

[0194] It should be understood that the processor can be a general-purpose processor, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0195] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random access memory (RAM). Memory is any medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited to this. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0196] Figure 15 This is another schematic diagram of a model-based signal inference device provided in an embodiment of this application. For example... Figure 15 As shown, the hardware architecture of the model-based signal inference device can be applied to SOC and application processor (AP) and can execute the model-based signal inference method provided in the embodiments of this application.

[0197] For example, the model-based signal inference device may include at least one CPU, at least one memory, a GPU, a decoder, a dedicated video / graphics processor, a receiving interface, and a transmitting interface. Optionally, the model-based signal inference device may also include a microprocessor and a microcontroller unit (MCU). In one optional case, the various parts of the model-based signal inference device are coupled together via connectors. It should be understood that in the various embodiments of this application, coupling refers to interconnection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, or buses. These interfaces are usually electrical communication interfaces, but mechanical interfaces or other forms of interfaces are not excluded. This embodiment does not limit this. In one optional case, the various parts are integrated on the same chip. In another optional case, the CPU, GPU, decoder, receiving interface, and transmitting interface can be integrated on a single chip, and the various parts inside the chip access external memory through a bus. The dedicated video / graphics processor can be integrated with the CPU on the same chip, or it can exist as a separate processor chip. For example, the dedicated video / graphics processor can be a dedicated image signal processor (ISP). The chip involved in this application embodiment is a system manufactured on the same semiconductor substrate using integrated circuit technology, also called a semiconductor chip. It can be a collection of integrated circuits formed on a substrate (usually a semiconductor material such as silicon) using integrated circuit technology, and its outer layer is typically encapsulated by semiconductor packaging materials. The integrated circuit can include various functional devices, each including logic gates, metal-oxide-semiconductor (MOS) transistors, bipolar transistors, or diodes, and may also include other components such as capacitors, resistors, or inductors. Each functional device can operate independently or under the action of necessary driving software, and can realize various functions such as communication, computation, or storage.

[0198] Optionally, the CPU can be a single-core or multi-core processor. Alternatively, the CPU can be a processor group consisting of multiple processors coupled to each other via one or more buses. In one optional scenario, the processing of image or video signals is partly performed by the GPU, partly by a dedicated video / graphics processor, and possibly by software code running on a general-purpose CPU or GPU.

[0199] The device may further include a memory for storing computer program instructions, including an operating system (OS), various user applications, and various types of computer program code, including program code for executing the present application. The memory may also be used to store video data, image data, etc. The CPU can be used to execute the computer program code stored in the memory to implement the methods in the embodiments of this application. Optionally, the memory may be non-volatile memory, such as an embedded multi-media card (EMMC), universal flash storage (UFS), or read-only memory (ROM), or other types of static storage devices capable of storing static information and instructions. It may also be volatile memory, such as RAM, or other types of dynamic storage devices capable of storing information and instructions. Alternatively, it may be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other computer-readable storage medium capable of carrying or storing program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0200] The receiving interface can be a data input interface for the processor chip. In one optional case, the receiving interface can be a mobile industry processor interface (MIPI), a high definition multimedia interface (HDMI), or a DisplayPort (DP), etc.

Claims

1. A model-based signal reasoning method, characterized in that, include: Receive an input signal, wherein the input signal is a digital signal that can be processed by a computer or processor; Based on the dependency information in the preset model, the input signal is inferred using the preset model to obtain the output signal; The preset model includes operators, each including input parameters and output parameters. The dependency information indicates the dependency relationship between the input parameters or output parameters of at least one operator in the target frame inference and the input parameters or output parameters of at least one operator in the reference frame inference when performing multi-frame inference on the input signal using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame inference among the other frames in the multi-frame inference besides the target frame inference. The step of inferring the input signal based on the dependency information in the preset model includes: Load the preset model and the dependency information; The execution order of each operator of the preset model in the current frame inference is determined based on the dependency information; According to the execution order, each operator of the preset model in the current frame inference is added to multiple execution queues; wherein, there is no dependency between the input parameters of the multiple operators executed at the same time in the multiple execution queues; Wherein, when the dependency information includes the dependency relationship between the input parameters of the operator in the target frame inference and the output parameters of the operator in the reference frame inference, the dependency information includes a first inter-frame interval, a first operator identifier, and a first parameter identifier. The first inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first input parameter of the first operator in the target frame inference. The first operator identifier is used to indicate the operator in the reference frame inference that is dependent on the first operator. The first parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first input parameter. When the dependency information includes the dependency relationship between the output parameters of the operator in the target frame inference and the input parameters of the operator in the reference frame inference, the dependency information includes a second inter-frame interval, a second operator identifier, and a second parameter identifier. The second inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first output parameter of the second operator in the target frame inference. The second operator identifier is used to indicate the operator in the reference frame inference that is dependent on the second operator, and the second parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first output parameter.

2. The method according to claim 1, characterized in that, Before performing multi-frame inference of the input signal based on the dependency information in the preset model, the method further includes: The dependency information is set in the description information of the preset model; wherein the description information is used to describe at least one operator included in the preset model and the input parameters and output parameters of each operator in the at least one operator.

3. The method according to claim 2, characterized in that, Setting the dependency information in the description information of the preset model includes: The dependency information is set in the description information of the preset model by calling the interface of the preset model. The interface of the preset model is used to provide an entry point for modifying information of the preset model.

4. The method according to claim 2, characterized in that, Setting the dependency information in the description information of the preset model includes: Receive the dependency information input by the user; The dependency information is set in the description information of the preset model based on the dependency information input by the user.

5. The method according to claim 2, characterized in that, Before setting the dependency information in the description information of the preset model, the method further includes: Obtain the description information; The dependency information is obtained based on at least one operator included in the description information, as well as the input and output parameters of each operator.

6. The method according to claim 1, characterized in that, The processable digital signals include at least one of the following: image data, voice data, and text data.

7. A model-based signal inference device, characterized in that, include: Processor and transmission interface; The transmission interface is used to receive input signals, which are digital signals that can be processed by a computer or processor. The processor is used to perform inference on the input signal based on the dependency information in the preset model to obtain an output signal; The preset model includes operators, each including input parameters and output parameters. The dependency information indicates the dependency relationship between the input parameters or output parameters of at least one operator in the target frame inference and the input parameters or output parameters of at least one operator in the reference frame inference when performing multi-frame inference on the input signal using the preset model. The target frame inference is at least one frame in the multi-frame inference, and the reference frame inference is at least one frame inference among the other frames in the multi-frame inference besides the target frame inference. The processor is specifically used for: Load the preset model and the dependency information; The execution order of each operator of the preset model in the current frame inference is determined based on the dependency information; According to the execution order, each operator of the preset model in the current frame inference is added to multiple execution queues; wherein, there is no dependency between the input parameters of the multiple operators executed at the same time in the multiple execution queues; Wherein, when the dependency information includes the dependency relationship between the input parameters of the operator in the target frame inference and the output parameters of the operator in the reference frame inference, the dependency information includes a first inter-frame interval, a first operator identifier, and a first parameter identifier. The first inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first input parameter of the first operator in the target frame inference. The first operator identifier is used to indicate the operator in the reference frame inference that is dependent on the first operator. The first parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first input parameter. When the dependency information includes the dependency relationship between the output parameters of the operator in the target frame inference and the input parameters of the operator in the reference frame inference, the dependency information includes a second inter-frame interval, a second operator identifier, and a second parameter identifier. The second inter-frame interval is used to indicate the inter-frame interval between the reference frame inference and the target frame inference, which is dependent on the first output parameter of the second operator in the target frame inference. The second operator identifier is used to indicate the operator in the reference frame inference that is dependent on the second operator, and the second parameter identifier is used to indicate the parameter in the reference frame inference that is dependent on the first output parameter.

8. The apparatus according to claim 7, characterized in that, The processor is also used for: The dependency information is set in the description information of the preset model; wherein the description information is used to describe at least one operator included in the preset model and the input parameters and output parameters of each operator in the at least one operator.

9. The apparatus according to claim 8, characterized in that, The processor is specifically used for: The dependency information is set in the description information of the preset model by calling the interface of the preset model. The interface of the preset model is used to provide an entry point for modifying information of the preset model.

10. The apparatus according to claim 8, characterized in that, The transmission interface is also used to receive the dependency information input by the user; The processor is specifically used to set the dependency information in the description information of the preset model according to the dependency information input by the user.

11. The apparatus according to claim 8, characterized in that, The processor is also used for: Obtain the description information; The dependency information is obtained based on at least one operator included in the description information, as well as the input and output parameters of each operator.

12. The apparatus according to any one of claims 7 to 11, characterized in that, The processable digital signals include at least one of the following: image data, voice data, and text data.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer or processor, implement the model-based signal reasoning method as described in any one of claims 1 to 6.

Images