Model reasoning method and device, electronic equipment and storage medium

Abstract

The invention provides a model reasoning method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. Inputting the input vector into a target model to obtain a model reasoning result corresponding to the input vector output by the target model; wherein the target model comprises at least one hybrid expert module; the hybrid expert module comprises a time sequence feature extraction layer, a routing selection layer, an expert model layer and a result output layer which are connected in sequence; the expert model layer comprises a plurality of state space models; the input vector is generated based on at least one of text, speech, image, or video. According to the method and the device provided by the invention, the storage consumption of the large model is reduced, the access frequency of the large model to the storage is reduced, and the reasoning efficiency of the large model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model reasoning method, device, electronic device and storage medium. Background Art

[0002] Large Language Model (LLM), also known as large model, refers to a machine learning model with a large parameter scale and complex model structure. Large models can capture and learn complex patterns and relationships in data, and can perform well in a variety of tasks, such as language translation, image recognition, natural language understanding and generation, etc.

[0003] The hybrid expert architecture is a special large model architecture that aims to improve efficiency and flexibility by decomposing a large model into multiple smaller sub-models (i.e., experts). Related technologies use feedforward neural networks or attention networks to build hybrid expert architectures. When processing long texts or image videos and other data, such hybrid expert architectures usually take up a lot of storage resources to save all historical information, and the reasoning efficiency is limited by hardware resources.

[0004] Therefore, how to reduce the consumption of storage resources by large models and improve the reasoning efficiency of large models has become a technical problem that needs to be solved urgently in the industry. Summary of the invention

[0005] The present application provides a model reasoning method, device, electronic device and storage medium, which are used to solve the technical problem of how to reduce the consumption of storage resources by large models and improve the reasoning efficiency of large models.

[0006] This application provides a model reasoning method, including: Get the input vector; Inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model; Among them, the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer includes multiple state space models; the input vector is generated based on at least one of text, speech, image or video.

[0007] In some embodiments, inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model includes: Inputting the input vector into the time series feature extraction layer to obtain a time series feature vector output by the time series feature extraction layer; Inputting the time series feature vector into the routing selection layer to obtain the selection results and output weights of each state space model output by the routing selection layer; Based on the selection results of each state space model, a target state space model is selected in the expert model layer, and the time series feature vector is input into each target state space model to obtain the expert calculation results output by each target state space model; The expert calculation results output by each target state space model and the output weights of each target state space model are input into the result output layer to obtain the model reasoning results output by the result output layer.

[0008] In some embodiments, the state-space model includes an input linear layer, a parameter calculation layer, a state calculation layer, a gating calculation layer, and an output linear layer.

[0009] In some embodiments, the step of inputting the time series feature vector into each target state space model to obtain the expert calculation results output by each target state space model includes: Inputting the time series feature vector into the input linear layer to obtain a first vector output by the input linear layer; Inputting the first vector into the parameter calculation layer to obtain the state parameter output by the parameter calculation layer; Inputting the first vector and the state parameter into the state calculation layer to obtain a second vector output by the state calculation layer; Inputting the first vector and the second vector into the gated computing layer to obtain a third vector output by the gated computing layer; The third vector is input into the output linear layer to obtain the expert calculation result output by the output linear layer.

[0010] In some embodiments, the expert model layers are deployed in parallel in different computing devices in a distributed computing system.

[0011] In some embodiments, the temporal feature extraction layer is constructed based on at least one of a recurrent neural network, a convolutional neural network, a temporal convolutional network, a long short-term memory network, and a gated recurrent unit.

[0012] The present application provides a model reasoning device, comprising: An acquisition unit, used for acquiring an input vector; An inference unit, used for inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model; Among them, the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer includes multiple state space models; the input vector is generated based on at least one of text, speech, image or video.

[0013] The present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the model reasoning method when executing the computer program.

[0014] The present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements the model reasoning method when executed by a processor.

[0015] The present application provides a computer program product, including a computer program, which implements the model reasoning method when executed by a processor.

[0016] The model reasoning method, device, electronic device and storage medium provided in the present application obtain an input vector; input the input vector into a target model to obtain a model reasoning result corresponding to the input vector output by the target model; the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected timing feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer adopts a state space model; since the state space model is used as an expert, only the state variables at a single moment need to be saved, and the requirements for storage and access are relatively low, which reduces the storage consumption of the large model, reduces the number of storage accesses of the large model, and improves the reasoning efficiency of the large model; since the timing feature extraction layer is set in the hybrid expert module, shared historical state information can be provided for each state space model, so that each state space model can perform calculations at the current moment based on the shared historical state information regardless of whether it was selected at the previous moment, thereby improving the reasoning accuracy of the large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flowchart of the model reasoning method provided in this application.

[0020] Figure 2 It is a structural diagram of the hybrid expert module provided in this application.

[0021] Figure 3 It is a structural diagram of the state space model provided in this application.

[0022] Figure 4 It is a structural diagram of the model reasoning device provided in this application.

[0023] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second" etc. in the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units or modules need not be limited to those steps or units or modules clearly listed, but may include other steps or units or modules that are not clearly listed or inherent to these processes, methods, products or equipment.

[0026] Related technologies use feedforward neural networks (FFN) or attention networks to build hybrid expert architectures to build large models. Both feedforward neural networks and attention networks require a large amount of storage resources to save all historical information when processing long texts or image and video data, and the reasoning efficiency is limited by hardware resources.

[0027] In order to solve the shortcomings of related technologies, Figure 1 It is a flowchart of the model reasoning method provided in this application, such as Figure 1As shown, the method includes step 110 and step 120.

[0028] Step 110: Obtain an input vector.

[0029] Specifically, the execution subject of the model reasoning method provided in the embodiment of the present application is a model reasoning device. The device can be implemented by software, such as a model reasoning program; it can also be implemented by hardware, such as various types of graphics processors (Graphics Processing Unit, GPU) and general-purpose graphics processors (General-Purpose Graphics Processing Unit, GPGPU) that execute the model reasoning method, as well as mobile terminals, tablet computers, desktop computers or servers containing these processors.

[0030] The application scenarios of the model reasoning method provided in the embodiments of the present application may be scenarios such as using large models for text processing, image processing, or speech processing.

[0031] The input vector refers to the vector representation obtained after processing the file that will be input into the large model for model inference. The input vector can be represented as multiple feature units (tokens). When the large model is used for image processing, the feature unit (token) can refer to an area or feature in the image. For example, an image block obtained by segmenting an image can be used as a feature unit. When the large model is used for speech processing, the feature unit (token) can refer to a segment or feature in the speech signal. For example, an audio frame obtained after segmenting a speech signal. When the large model is used for natural language understanding or text generation, the feature unit (token) can refer to the smallest semantic unit in the text.

[0032] The model reasoning method provided in this application is described using text processing as a specific application scenario. The input of the large model can be input text. The input text is preprocessed, segmented, embedded, and positionally encoded to obtain an input vector corresponding to the input text.

[0033] Step 120: input the input vector into the target model to obtain the model inference result corresponding to the input vector output by the target model; wherein the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected timing feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer adopts a state space model.

[0034] Specifically, a target model can be constructed, and the input vector can be processed to obtain the model inference result. In the text processing scenario, the model inference result can be generated content, text classification, knowledge question answering, and translation results.

[0035] The target model is a neural network model that uses a mixed expert architecture, such as a large model such as the Transformer model and the BERT (Bidirectional Encoder Representations from Transformers) model. The Mixture of Experts (MoE) module is a neural network model or network module that performs model calculations in the target model.

[0036] The state space model (Super Space Model, SSM) is used to describe dynamic systems and is widely used in control theory, signal processing, and time series analysis. It can capture the characteristics of the system that change over time. Its basic function is to Input With the previous moment System status , calculate and predict the next moment based on the system parameter matrices A, B, C and D Output , which can be expressed as:

[0037] In the formula, For the current moment system status.

[0038] Related technologies use feedforward neural networks or attention networks to build hybrid expert architectures. Taking the use of attention networks to build hybrid expert architectures as an example, attention networks can save longer memories (historical state information), but because they need to save state information at each moment, they require a larger key-value cache (KV cache), which has higher requirements for storage and access. The state space model is calculated in a recursive form, and the current calculation result only depends on the current input and the system state at the previous moment. It only needs to save the state variables at a single moment, and has lower requirements for storage and access. Therefore, the consumption of storage resources by large models can be reduced.

[0039] In order to capture the temporal features in sequence data, effectively extract the local dependencies in time, and better understand the patterns of data changes over time, the state space model and temporal feature extraction can be combined. This combination is particularly important when processing time series data or dynamic data such as video frames, and can enhance the target model's ability to model temporal information.

[0040] Since it is necessary to construct a hybrid expert module based on multiple state space models, if a state space model is not selected at a certain moment, an activation signal at the corresponding moment will not be generated. When the state space model is selected and time series feature extraction is required at the next moment, there is no historical state information to complete the operation. In order to solve this technical problem, the hybrid expert module provided in the embodiment of the present application includes a time series feature extraction layer, a routing selection layer, an expert model layer and a result output layer connected in sequence.

[0041] Among them, the time series feature extraction layer is a time series feature extraction layer shared by all experts in the expert model layer. In each iterative calculation during the model reasoning process, regardless of whether each expert is selected, the time series feature extraction layer participates in the calculation, and can extract the state information of the input vector at each moment, and provide each expert with shared historical state information, so that each expert can calculate the system state at the current moment and predict the output at the next moment based on the historical state information, regardless of whether they were selected at the previous moment.

[0042] The routing selection layer is used to dynamically select the most appropriate expert for processing according to the characteristics of the input when processing the input, thereby improving efficiency and performance.

[0043] The expert model layer uses multiple state-space models as experts, each of which is optimized on a specific task or data feature and has different capabilities and expertise.

[0044] The result output layer is used to fuse the outputs of various experts to obtain the model inference results.

[0045] The model reasoning method provided in the embodiment of the present application obtains an input vector; inputs the input vector into the target model to obtain a model reasoning result corresponding to the input vector output by the target model; the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected timing feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer adopts a state space model; since the state space model is used as an expert, only the state variables at a single moment need to be saved, and the requirements for storage and access are relatively low, which reduces the storage consumption of the large model, reduces the number of storage accesses of the large model, and improves the reasoning efficiency of the large model; since the timing feature extraction layer is set in the hybrid expert module, shared historical state information can be provided for each state space model, so that each state space model can perform calculations at the current moment based on the shared historical state information regardless of whether it was selected at the previous moment, thereby improving the reasoning accuracy of the large model.

[0046] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0047] In some embodiments, inputting an input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model includes: Input the input vector into the time series feature extraction layer to obtain the time series feature vector output by the time series feature extraction layer; Input the time series feature vector into the routing selection layer to obtain the selection results and output weights of each state space model output by the routing selection layer; Based on the selection results of each state space model, the target state space model is selected in the expert model layer, the time series feature vector is input into each target state space model, and the expert calculation results output by each target state space model are obtained; The expert calculation results output by each target state space model and the output weights of each target state space model are input into the result output layer to obtain the model reasoning results output by the result output layer.

[0048] Specifically, the model reasoning method provided in this application is described below by taking the target model including a hybrid expert module as an example.

[0049] Figure 2 It is a structural diagram of the hybrid expert module provided by this application, such as Figure 2 As shown, the target model 200 includes a hybrid expert module 210; the hybrid expert module 210 includes a sequentially connected temporal feature extraction layer 211, a routing selection layer 212, an expert model layer 213 and a result output layer 214.

[0050] The input vector is input into the temporal feature extraction layer to obtain the temporal feature vector output by the temporal feature extraction layer. The temporal feature extraction layer can be a separate network layer, such as a temporal convolutional layer; or it can be a neural network, such as a recurrent neural network (RNN), a convolutional neural network (CNN), a temporal convolutional network (TCN), a long short-term memory network (LSTM), and a gated recurrent unit (GRU).

[0051] The time series feature vector is input into the routing selection layer to obtain the selection results and output weights of each state space model output by the routing selection layer. The routing selection layer can be implemented by a routing network (Gating Network), which can be implemented by a Softmax activation function and a classification network. The routing selection layer can also be implemented by a fully connected layer (DenseLayer) or an attention mechanism. The routing selection layer can use the TOP-K (select the top K elements from a set) algorithm. The value of K can be set as needed.

[0052] The selection result is used to indicate the selection result of each expert in the expert model layer by the routing selection layer. The output weight is used to indicate the influence of the calculation result of the selected expert on the model reasoning result.

[0053] According to the selection results of each state space model, the target state space model is selected in the expert model layer. The target state space model refers to the expert selected in the expert model layer. By inputting the time series feature vector into each target state space model respectively, the expert calculation results output by each target state space model can be obtained.

[0054] The expert calculation results output by each target state space model and the output weights of each target state space model are input into the result output layer, and the result output layer performs weighted summation on the expert calculation results according to the output weights to obtain the model inference results output by the result output layer.

[0055] The model reasoning method provided in the embodiment of the present application inputs the input vector into the target model, which is processed by the hybrid expert module in the target model to obtain the model reasoning result corresponding to the input vector. Different expert models can be combined to perform calculations under limited computing resources, and the advantages of each expert model are utilized to improve the reasoning efficiency of large models.

[0056] In some embodiments, the state-space model includes an input linear layer, a parameter computation layer, a state computation layer, a gating computation layer, and an output linear layer.

[0057] Specifically, the expert model layer includes multiple state-space models arranged in parallel. Figure 3 It is a structural diagram of the state space model provided by this application, such as Figure 3 As shown, the state-space model 300 includes an input linear layer 310 , a parameter calculation layer 320 , a state calculation layer 330 , a gating calculation layer 340 , and an output linear layer 350 .

[0058] The input linear layer is used to perform linear transformation on the input vector. The parameter calculation layer is used to calculate the parameters of the state space model, such as the state transfer matrix and the observation matrix. The state calculation layer is used to calculate and predict the output at the next moment based on the input at the current moment and the system state at the previous moment, as well as the system parameter matrix. The gate calculation layer is used to dynamically select or adjust the output. The output linear layer is used to perform linear transformation on the output result.

[0059] According to the structure of the hybrid expert module and the state-space model provided in the embodiment of the present application, the time series feature extraction layer in the hybrid expert module is before the expert model layer, and the expert model layer includes the state-space model, and the state-space model includes the input linear layer. It can be determined that the input linear layer is located after the time series feature extraction layer.

[0060] By setting the time series feature extraction layer in front and the input linear layer in the back, on the one hand, it is convenient for various state-space models to share the time series feature extraction layer, so that each state-space model can perform calculations at the current moment based on the shared historical state information regardless of whether it was selected at the previous moment; on the other hand, setting the input linear layer in the state-space model facilitates the implementation of functions such as feature mapping, dimensionality adjustment and feature extraction, and there is no need to set a separate linear layer in the hybrid expert module. Compared with the structure of the Mamba module in related technologies, it avoids model structure redundancy and reduces the computational cost and implementation difficulty of model training and reasoning.

[0061] The model reasoning method provided in the embodiment of the present application, the state space model includes an input linear layer, a parameter calculation layer, a state calculation layer, a gating calculation layer and an output linear layer, which can effectively describe and predict the output results and improve the reasoning efficiency of large models.

[0062] In some embodiments, the time series feature vector is input into each target state space model to obtain the expert calculation results output by each target state space model, including: Input the time series feature vector into the input linear layer to obtain the first vector output by the input linear layer; Input the first vector into the parameter calculation layer to obtain the state parameter output by the parameter calculation layer; Input the first vector and the state parameter into the state calculation layer to obtain a second vector output by the state calculation layer; Input the first vector and the second vector into the gated computing layer to obtain a third vector output by the gated computing layer; The third vector is input into the output linear layer to obtain the expert calculation result output by the output linear layer.

[0063] Specifically, the time series feature vector output by the time series feature extraction layer is input into the input linear layer in the target state space model, and the input linear layer performs a linear transformation to obtain a first vector.

[0064] The first vector is input into the parameter calculation layer, and the parameter calculation layer calculates the state parameters of the state space model, such as the system parameter matrices A, B, C and D in the above embodiment.

[0065] The first vector and the state parameter are input into the state calculation layer to obtain the second vector output by the state calculation layer. The second vector can be the output at the next moment predicted by the target state space model.

[0066] The first vector and the second vector are input into the gated computing layer to obtain the third vector output by the gated computing layer. The gated computing layer controls the output of the state space model through the gating mechanism. The gated computing layer dynamically controls the information flow and improves the expressiveness of the model by selectively retaining or suppressing the input signal. It can effectively capture long-distance dependencies, prevent gradient disappearance, and help the model focus on important features or states.

[0067] Finally, the third vector is input into the output linear layer, which performs linear transformation to obtain the expert calculation result output by the target state space model.

[0068] The model reasoning method provided in the embodiment of the present application inputs the time series feature vector into each target state space model to obtain the expert calculation results output by each target state space model, thereby improving the reasoning efficiency of the large model.

[0069] In some embodiments, expert model layers are deployed in parallel in different computing devices in a distributed computing system.

[0070] Specifically, considering that the target model has a complex structure and many parameters, the target model can be deployed in a distributed computing system.

[0071] A distributed computing system may include multiple computing devices arranged in parallel, which are nodes in the distributed computing system. For example, a distributed computing system may be a cluster of multiple servers connected through a network, working together to provide higher availability, reliability, and performance. Distributed computing systems can share loads, achieve failover and load balancing, and are suitable for large-scale applications, data processing, and scenarios with high availability requirements.

[0072] Since each state-space model in the expert model layer is set up in parallel, each state-space model can be deployed in parallel on different computing devices in a distributed computing system, which can significantly improve computing efficiency and model performance. By distributing different state-space models to multiple computing devices, load balancing can be achieved and the computing pressure of a single device can be reduced. In addition, the use of a model parallel strategy can speed up training and inference speeds, thereby better processing large-scale data sets. This architecture is also easy to expand to adapt to growing computing needs.

[0073] The model reasoning method provided in the embodiment of the present application deploys the expert model layer in parallel in different computing devices in a distributed computing system, which can significantly improve computing efficiency and model performance.

[0074] In some embodiments, the input vector is generated based on at least one of text, speech, image, or video.

[0075] Specifically, the model reasoning method provided in the embodiment of the present application is suitable for processing multimodal data, including text, voice, image or video.

[0076] Taking text generation as an example, by performing preprocessing, word segmentation, word embedding, position encoding and other steps on the input text, we can obtain the input vector corresponding to the input text.

[0077] The input vector is input into the target model, and the target model inputs the input vector into the hybrid expert module. The time series feature extraction layer extracts the time series features of the input vector to obtain the time series feature vector. The routing selection layer selects K state space models in the expert model layer as the target state space model based on the time series feature vector and uses the TOP-K algorithm, and outputs the output weights of each target state space model. In the expert model layer, each target state space model calculates the time series feature vector and outputs the expert calculation results. The result output layer calculates the model inference result based on the expert calculation results output by each target state space model and the output weights of each target state space model.

[0078] The target model may have multiple hybrid expert modules, and may also have network modules in other related technologies. The embodiment of the present application does not specifically limit the specific model structure of the target model.

[0079] The model inference method provided in the embodiment of the present application, the target model can process multimodal data and has strong applicability.

[0080] The present application proposes to construct a hybrid expert module with a state-space model as a single structure. A large model can be constructed by stacking hybrid expert modules. This is simpler than the implementation structure in the related technology, and can make full use of the information capacity of the parameters of the state-space model module itself, thereby improving the overall parameter utilization efficiency of the model.

[0081] The hybrid state space model expert layer of the present application decouples the temporal convolution and other functional modules in the state space model structure, and the improved state space model expert module can be easily expanded from the width direction by replication. In addition, the calculation of each expanded part is independent and not limited by the original temporal convolution, which can more effectively utilize the parallel computing capabilities of various graphics processing units (GPUs).

[0082] The device provided in the embodiment of the present application is described below. The device described below and the method described above can be referenced to each other.

[0083] Figure 4 is a schematic diagram of the structure of the model reasoning device provided by this application, such as Figure 4 As shown, the device comprises: An acquisition unit 410, configured to acquire an input vector; The inference unit 420 is used to input the input vector into the target model and obtain the model inference result corresponding to the input vector output by the target model; The target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer adopts a state space model.

[0084] The model inference device provided in the embodiment of the present application obtains an input vector; inputs the input vector into the target model to obtain a model inference result corresponding to the input vector output by the target model; the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer adopts a state space model; since the state space model is used as an expert, only the state variables at a single moment need to be saved, and the requirements for storage and access are relatively low, which reduces the storage consumption of the large model, reduces the number of storage accesses of the large model, and improves the inference efficiency of the large model; since the temporal feature extraction layer is set in the hybrid expert module, shared historical state information can be provided for each state space model, so that each state space model can perform calculations at the current moment based on the shared historical state information regardless of whether it was selected at the previous moment, thereby improving the inference accuracy of the large model.

[0085] In some embodiments, the inference unit is configured to: Input the input vector into the time series feature extraction layer to obtain the time series feature vector output by the time series feature extraction layer; Input the time series feature vector into the routing selection layer to obtain the selection results and output weights of each state space model output by the routing selection layer; Based on the selection results of each state space model, the target state space model is selected in the expert model layer, the time series feature vector is input into each target state space model, and the expert calculation results output by each target state space model are obtained; The expert calculation results output by each target state space model and the output weights of each target state space model are input into the result output layer to obtain the model reasoning results output by the result output layer.

[0086] In some embodiments, the state-space model includes an input linear layer, a parameter computation layer, a state computation layer, a gating computation layer, and an output linear layer.

[0087] In some embodiments, the inference unit is configured to: Input the time series feature vector into the input linear layer to obtain the first vector output by the input linear layer; Input the first vector into the parameter calculation layer to obtain the state parameter output by the parameter calculation layer; Input the first vector and the state parameter into the state calculation layer to obtain a second vector output by the state calculation layer; Input the first vector and the second vector into the gated computing layer to obtain a third vector output by the gated computing layer; The third vector is input into the output linear layer to obtain the expert calculation result output by the output linear layer.

[0088] In some embodiments, expert model layers are deployed in parallel in different computing devices in a distributed computing system.

[0089] In some embodiments, the input vector is generated based on at least one of text, speech, image, or video.

[0090] Figure 5 is a schematic diagram of the structure of the electronic device provided by this application, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus (Communications Bus) 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic command in the memory 530 to execute the method described in the above embodiment, for example: Obtain an input vector; input the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model; wherein the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer includes a plurality of state space models; the input vector is generated based on at least one of text, speech, image or video.

[0091] In addition, the logical commands in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0092] The processor in the electronic device provided in the embodiment of the present application can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.

[0093] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the methods provided in the above embodiments are implemented.

[0094] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.

[0095] An embodiment of the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model reasoning method, characterized in that: include: Get the input vector; Inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model; Among them, the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer includes multiple state space models; the input vector is generated based on at least one of text, speech, image or video.

2. The model reasoning method according to claim 1, characterized in that: The step of inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model includes: Inputting the input vector into the time series feature extraction layer to obtain a time series feature vector output by the time series feature extraction layer; Inputting the time series feature vector into the routing selection layer to obtain the selection results and output weights of each state space model output by the routing selection layer; Based on the selection results of each state space model, a target state space model is selected in the expert model layer, and the time series feature vector is input into each target state space model to obtain the expert calculation results output by each target state space model; The expert calculation results output by each target state space model and the output weights of each target state space model are input into the result output layer to obtain the model reasoning results output by the result output layer.

3. The model reasoning method according to claim 2, characterized in that: The state space model includes an input linear layer, a parameter calculation layer, a state calculation layer, a gating calculation layer and an output linear layer.

4. The model reasoning method according to claim 3, characterized in that: The step of inputting the time series feature vector into each target state space model to obtain the expert calculation results output by each target state space model includes: Inputting the time series feature vector into the input linear layer to obtain a first vector output by the input linear layer; Inputting the first vector into the parameter calculation layer to obtain the state parameter output by the parameter calculation layer; Inputting the first vector and the state parameter into the state calculation layer to obtain a second vector output by the state calculation layer; Inputting the first vector and the second vector into the gated computing layer to obtain a third vector output by the gated computing layer; The third vector is input into the output linear layer to obtain the expert calculation result output by the output linear layer.

5. The model reasoning method according to any one of claims 1 to 4, characterized in that: The expert model layers are deployed in parallel in different computing devices in a distributed computing system.

6. The model reasoning method according to any one of claims 1 to 4, characterized in that: The temporal feature extraction layer is constructed based on at least one of a recurrent neural network, a convolutional neural network, a temporal convolutional network, a long short-term memory network and a gated recurrent unit.

7. A model reasoning device, characterized in that: include: An acquisition unit, used for acquiring an input vector; An inference unit, used for inputting the input vector into a target model to obtain a model inference result corresponding to the input vector output by the target model; Among them, the target model includes at least one hybrid expert module; the hybrid expert module includes a sequentially connected temporal feature extraction layer, a routing selection layer, an expert model layer and a result output layer; the expert model layer includes multiple state space models; the input vector is generated based on at least one of text, speech, image or video.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the model reasoning method described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model reasoning method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the model reasoning method according to any one of claims 1 to 6 is implemented.

Images