Model calling method and device, computer device and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a model calling method and device, computer equipment and a storage medium. The method determines a demand model type through comparison of a request website and a type string, determines a target decoding mode corresponding to the demand model type from a decoding mode set, decodes received encoded request data by using the target decoding mode, obtains a model name, determines a target channel from a memory mapping according to the model name, sends the request data to the target channel and uses a target model to perform calculation, returns a calculation result to a calling party through the target channel, directly determines a corresponding target model according to the model name, avoids a situation that a model is not adapted, improves the efficiency of model calling, and the calling party and the model are completely decoupled. When the model changes, the target channel can still be distributed through the model name, the communication and coordination cost of a model maintenance party and the calling party is saved, and the efficiency of model calling is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a model calling method and device, a computer device and a storage medium. BACKGROUND

[0002] Due to the huge storage space and computing resources required by complex network models, conventional computer devices are difficult to meet the needs of complex network models. At present, the utilization rate of network models is improved by using the model calling method. The complex network model is deployed in a server with high computing performance. Each calling party calls the server by requesting a website, and then sends the data to be processed to the server. The server performs a computing process and returns the computing result to the calling party.

[0003] Generally, the format of the request website is fixed. When there are many model types in the server, it is difficult for the calling party to distinguish the models. Therefore, one solution is to specify an access domain name for each model in the server to access the model. However, when the model changes or migrates, the corresponding access domain name will also change. At this time, the calling party needs to be notified to modify the corresponding access domain name, which reduces the efficiency of model calling. Therefore, how to improve the efficiency of model calling has become a problem to be solved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a model calling method and device, a computer device and a storage medium to solve the problem of low efficiency of model calling.

[0005] In a first aspect, the embodiments of the present application provide a model calling method, which comprises:

[0006] obtaining a request website and encoded request data of a calling party, comparing the request website with a preset type string by string, and if the comparison result is consistent, determining that the type string is a demand model type;

[0007] According to the demand model type, a target decoding mode corresponding to the demand model type is determined from a preset decoding mode database, and the encoded request data is decoded according to the target decoding mode to obtain a model name;

[0008] According to the model name, a target channel is determined from a preset memory mapping, and the request data is sent to the target channel. The memory mapping includes a correspondence between the model name and the channel, and the target channel is used to connect the calling party and the loaded target model;

[0009] The request data is input into the loaded target model for calculation, and the calculation result is returned to the calling party through the target channel.

[0010] In a second aspect, an embodiment of the present application provides a model calling device, the model calling device comprising:

[0011] a type determining module configured to obtain a request website and encoded request data of a calling party, and perform string comparison between the request website and a preset type string, and determine the type string as a demand model type if the comparison result is consistent;

[0012] a data decoding module configured to determine a target decoding mode corresponding to the demand model type from a preset decoding mode database according to the demand model type, and decode the encoded request data according to the target decoding mode to obtain a model name;

[0013] a channel determining module configured to determine a target channel from a preset memory mapping according to the model name, and send the request data to the target channel, wherein the memory mapping comprises a correspondence between a model name and a channel, and the target channel is configured to connect the calling party and a loaded target model;

[0014] a model calling module configured to input the request data into the loaded target model for calculation, and return a calculation result to the calling party through the target channel.

[0015] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the model calling method according to the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, and the computer program implements the model calling method according to the first aspect when executed by a processor.

[0017] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0018] The request website and the encoded request data of the calling party are acquired, the request website is compared with a preset type string, if the comparison result is consistent, it is determined that the type string is a demand model type, a target decoding mode corresponding to the demand model type is determined from a preset decoding mode set according to the demand model type, the request data is decoded according to the target decoding mode, a model name is obtained, a target channel is determined from a preset memory mapping according to the model name, the request data is sent to the target channel, the target channel is used for connecting the calling party and the target model, the request data is input into the target model for calculation, and the calculation result is returned to the calling party through the target channel, the demand model type is obtained by parsing the request website, the corresponding target structure definition is determined to decode the request data, the model name is obtained, and then the target model and the target channel of the calling party are determined according to the model name, the corresponding target model can be directly determined according to the model name, the situation that the model is not adapted is avoided, the efficiency of model calling is improved, and the calling party and the model are completely decoupled, when the model changes, the target channel can still be distributed through the model name, the communication and coordination cost of the model maintenance party and the calling party is saved, and the efficiency of model calling is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is an application environment schematic diagram of a model calling method provided by the first embodiment of the present application;

[0021] Figure 2 is a flow schematic diagram of a model calling method provided by the first embodiment of the present application;

[0022] Figure 3 is a flow schematic diagram of a model calling method provided by the second embodiment of the present application;

[0023] Figure 4 is a structure schematic diagram of a model calling device provided by the third embodiment of the present application;

[0024] Figure 5 is a structure schematic diagram of a computer device provided by the fourth embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0026] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, include the presence of one or more features, integers, steps, operations, elements, and / or components described in the specification, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It will be understood that the term "and / or," when used in the specification and in the following claims, is intended to mean one or more of the associated listed items can be present, and includes the possibilities of one or more of the associated listed items being present, and all possible combinations of one or more of the associated listed items.

[0028] As used in the description of the application and the following claims, the term "if" can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon [the described condition or event] being detected" or "in response to [the described condition or event] being detected," depending on the context.

[0029] In addition, the description in the specification of the application and the appended claims, the terms "first," "second," "third," etc. are used merely as labels, and are not intended to impose numerical or sequential order unless it is clearly indicated by context. These terms are also used in the description in a relative sense and for the purpose of distinguishing between various features.

[0030] Reference in the specification to "one embodiment" or "an embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, although it can. Furthermore, the terms "comprises," "comprising," "includes," "including," "has," "having" and the like are used synonymously to denote or describe the presence of the stated feature, integer, step, operation, element, component, or the like along with any associated conceivable

[0031] It should be understood that the size of the serial number of each step in the following embodiment does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0032] In order to illustrate the technical solutions of the present application, the following will be illustrated by specific embodiments.

[0033] The model calling method provided by the embodiment of the present application can be applied in the application environment such as Figure 1 , wherein the server and the client communicate. The client includes but is not limited to a palm computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud terminal device, a personal digital assistant (PDA) and the like. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0034] Referring to Figure 2 , it is a flowchart of the model calling method provided by the embodiment of the present application. The above model calling method can be applied to the server in Figure 1 , the server is connected with the computer device corresponding to the client to obtain the request URL and the encoded request data sent by the calling party. The calling party can refer to the user of the client. The server internally deploys at least one loaded model to provide model calling service for the calling party. As shown in Figure 2 , the model calling method can include the following steps:

[0035] Step S201, obtaining the request URL and the encoded request data of the calling party, and performing string comparison on the request URL and the preset type string. If the comparison result is consistent, it is determined that the type string is the demand model type.

[0036] The calling party can refer to a user who needs to perform model calling service. The user inputs the request of the model calling service through the computer device corresponding to the client, and the computer device corresponding to the client generates the request URL and the encoded request data. The request URL can refer to a uniform resource locator (URL). The request URL includes but is not limited to a protocol, a host name and a path. In this embodiment, the protocol can adopt gRPC protocol. The gRPC protocol is realized based on HTTP / 2 protocol. The host name can refer to the identification of the computer device corresponding to the client. The path can refer to the location of the encoded request data in the computer device corresponding to the client.

[0037] The HTTP / 2 protocol includes services, methods and data structures, the data structure can refer to a model type, the model type can refer to a framework to which the model belongs, the framework can include model building frameworks such as TensorFlow and Pytorch, the service can refer to content that needs to be executed by the called model, the service can include prediction services and GRPC inference services, and the method can refer to a processing method used when the called model executes the service, which can include prediction and model inference.

[0038] The encoded request data can include a model name and to-be-processed data, the to-be-processed data can refer to data that needs to be calculated by the called model, and the model name can refer to the specific name of the model, which can include prediction models, classification models, instance segmentation models, and semantic segmentation models.

[0039] The type string can refer to a preset model type string, the type string includes at least one model type string, and the type string is determined according to a model building framework to which a model deployed on a server belongs.

[0040] The string comparison can refer to finding a type string in a request URL, and the finding method varies according to different languages. For example, in Python, the find function, the index function, and the rfind function can be used for finding.

[0041] Specifically, the consistent comparison result can mean that a type string is found in a request URL, which means that the model type corresponding to the type string is the model type requested by the request URL, and the type string is determined as the demand model type.

[0042] Optionally, after comparing the request URL with the preset type string, the method further includes:

[0043] If the comparison result is inconsistent, the preset identification string is compared with the request URL to obtain the position of the identification string in the request URL.

[0044] According to the position and the preset separator, the to-be-updated string is extracted from the request URL, the update decoding mode corresponding to the to-be-updated string is determined from the decoding mode set, the request data is decoded according to the update decoding mode, and the model name is obtained.

[0045] When it is detected that the memory mapping contains the model name, the to-be-updated string is updated to the type string.

[0046] The preset type string contains at least one type string, and the inconsistent comparison result can mean that any type string is not found in the request URL.

[0047] The identification string can refer to identification information corresponding to a service, a method, a data structure, etc., for example, the identification of a data structure is package, the identification of a service is service, and the identification of a method is method. The position can refer to the position of the starting character of the identification string in the request URL based on the separator, for example, the character position can be the first character after the second separator in the request URL.

[0048] The preset separator can refer to a separator character between different string contents, for example, the separator character can be a punctuation mark, and the punctuation mark can refer to a period, a semicolon, etc. The string to be updated can refer to string information that needs to be updated to a type string.

[0049] The update decoding mode can refer to the decoding mode of the string to be updated. The model name can include a prediction model, a classification model, an instance segmentation model, a semantic segmentation model, etc. The memory mapping can refer to a collection of description information corresponding to each deployed model on the server. The description information can include a model name, a physical access address, etc. The physical access address can refer to the physical address when accessing the model on the server.

[0050] Specifically, the request URL is like " / package.service / method", the identification character is a period and a slash, and the identification string package is not after any separator, so the position of the identification string package in the request URL is the first character. The identification character service is after the first separator, so the position of the identification string service in the request URL is the first character after the first separator.

[0051] Since the position of the identification string is actually the position of the starting character of the identification string, the string between the position of the identification string and the next separator is the string to be updated, that is, the string to be updated starts with the first character after a separator and ends with the first character before the next separator.

[0052] The embodiment is applicable to a case where the server has deployed a model belonging to a model building framework, but the preset type string does not yet contain a type string corresponding to the model building framework. Since the model update and the type string update are not coupled, the type string update may be lagging behind. At this time, the type string is updated according to the manner of the embodiment.

[0053] The memory mapping has a coupling relationship with the model deployed on the server, that is, when the model deployed on the server is updated, the memory mapping is updated accordingly to ensure that the model can be normally accessed.

[0054] When it is detected that the model name is contained in the memory mapping, it is indicated that the model to be called this time is contained in the server, but the type string corresponding to the model to be called has not been updated in the preset type string. At this time, the step of updating the to-be-updated string to the type string is performed.

[0055] It should be noted that, generally, when a new model building framework is deployed on the server, the corresponding decoding mode needs to be updated in the decoding mode set. Therefore, if the update decoding mode corresponding to the to-be-updated string cannot be found from the decoding mode set, it indicates that the decoding mode set is updated laggingly or the corresponding model building framework is not deployed on the server.

[0056] The embodiment determines the to-be-updated string from the request URL according to the identification string and the separator, and updates the to-be-updated string to the type string, which can avoid that after the model building framework on the server is updated, the new string corresponding to the request URL cannot be recognized by the preset type string, resulting in that the corresponding model calling cannot be performed. In addition, whether the model to be called is deployed is verified through the memory mapping, and after it is confirmed that the model to be called is deployed, the type string is updated, which can avoid that after the type string is directly updated, the request is responded but the model to be called cannot be found in the server, thereby reducing invalid responses and improving the efficiency of model calling.

[0057] The above steps of obtaining the request URL of the calling party and the encoded request data, and performing string comparison between the request URL and the preset type string, if the comparison result is consistent, determining that the type string is the demand model type, can determine the demand model type according to the request URL in advance, without the need to decode the encoded request data, thereby saving computing resources and improving the efficiency of model calling.

[0058] In step S202, the target decoding mode corresponding to the demand model type is determined from the preset decoding mode set according to the demand model type, and the encoded request data is decoded according to the target decoding mode to obtain the model name.

[0059] The decoding mode set can include at least one decoding mode, and the decoding mode corresponds to the demand model type. When the protocol is a gRPC protocol, the decoding mode is a protocol buffer (pb) decoding mode, but different pb decoding definition modes correspond to different demand model types. The target decoding mode can refer to the decoding definition mode corresponding to the demand model type.

[0060] The request data is encoded by the calling party according to the encoding mode corresponding to the demand model type to obtain the encoded request data. The purpose of encoding can include compressing transmission bytes, improving data security, and the like.

[0061] The step of determining the target decoding mode corresponding to the demand model type from the preset decoding mode set according to the demand model type, decoding the encoded request data according to the target decoding mode, and obtaining the model name can compress the transmission bytes of the data, improve the efficiency of data transmission, and enhance the security of the data transmission process.

[0062] In step S203, the target channel is determined from the preset memory mapping according to the model name, and the request data is sent to the target channel.

[0063] The memory mapping includes the correspondence between the model name and the channel. The preset memory mapping can include the physical access address of each model corresponding to the model name. The physical access address can refer to a port of the channel, that is, the memory mapping includes the correspondence between the model name and the channel port. The target channel can refer to a long connection. The target channel is used to connect the calling party and the loaded target model. The loaded target model can refer to a trained target model. The target model can refer to a model that needs to be called.

[0064] Specifically, the model port address of the target model is found in the memory mapping according to the model name, the model port address is connected with the request port address of the calling party to obtain the target channel, the target channel is only used for data transmission between the target model and the calling party, and the calling party sends the encoded request data to the target channel, that is, to the model port address through the target channel.

[0065] Optionally, the demand model type includes at least two model architecture types, and one model architecture type corresponds to one channel in the memory mapping.

[0066] Before determining the target channel from the preset memory mapping according to the model name, the method further includes:

[0067] The demand model types of the obtained at least two historical request websites are counted, and the historical request quantities of each model architecture type are obtained according to the counting result.

[0068] The historical request quantities of each model architecture type are spliced into a reference quantity vector according to the model architecture type dimension, a target proportion vector most similar to the reference quantity vector is determined from the preset traffic proportion vector, and the traffic of all channels is proportioned according to the target proportion vector.

[0069] The model architecture type can refer to a model construction framework type, the channel can be a model port, the historical request website can refer to a request website received before receiving the request sent by the calling party, and the historical request data can refer to all request data received by the server within a preset target time period.

[0070] The number of historical requests of a model architecture type can refer to the number of historical request URLs of the model architecture type in all historical request URLs, and an element in the reference quantity vector represents the number of historical request URLs of the corresponding model architecture type. The traffic distribution vector can refer to a preset reference traffic distribution mode of each channel, and the target distribution vector can refer to an actual traffic distribution mode of each channel. The traffic distribution is used to make the traffic of the target channel meet the calculation requirements of the request data.

[0071] For example, assuming that the number of historical request URLs is N, the number of categories of demand model types is C, and the number of historical requests of the cth demand model type is n c , that is, the sum of the number of historical requests of all demand model types is the number of historical request URLs, and the number of historical requests of each model architecture type is spliced into a reference quantity vector according to the model architecture type dimension. The reference quantity vector can be represented as [n1, n2, …, n c , …, n C ]. The size of the reference quantity vector is 1*C, that is, one row and C columns, and each column corresponds to a model architecture type.

[0072] Similarly, the traffic distribution vector also has a size of 1*C columns. The kth traffic distribution vector can be represented as and satisfies where M can refer to the maximum traffic of the server, k can take values in the range of [1, 2, …, K], K can be the total number of traffic distribution vectors, and in the similarity calculation, the similarity calculation method can use Manhattan distance, Euclidean distance, cosine similarity, etc. In this embodiment, cosine similarity is used for similarity calculation, that is, where α k can refer to the cosine similarity calculated by the kth traffic distribution vector and the reference quantity vector, n represents the vector [n1, n2, …, n c , …, n C ], and m k represents the term The greater the cosine similarity distance, the more similar the traffic distribution vector and the reference quantity vector, and the traffic distribution vector corresponding to max(α k ) is determined as the target matching vector.

[0073] In this embodiment, the number of historical requests of the model architecture type is used for traffic distribution of each channel, which plays a load balancing effect and avoids the situation that the server cannot effectively process other requests at the same period after determining the traffic of the target channel, thereby effectively improving the efficiency of model invocation.

[0074] Optionally, after splicing all historical request quantities according to the model architecture type dimension into the reference quantity vector, the following steps are included:

[0075] obtaining an average processing duration of each model architecture type, multiplying the historical request quantity of the corresponding model architecture type and the average processing duration of the corresponding model architecture type to obtain a multiplication result of the corresponding model architecture type for any model architecture type;

[0076] concatenating the multiplication result of each model architecture type into a reference quantity vector according to the model architecture type dimension.

[0077] The average processing duration of a model architecture type can be the mean of the processing durations corresponding to all requests of the model architecture type in the target time period.

[0078] Specifically, the historical request quantity of the model architecture type and the average processing duration of the corresponding model architecture type are multiplied, the historical request quantity of the cth model architecture type is n c , and the average processing duration of the cth model architecture type is t c , then the multiplication result is n c *t c , the multiplication result of each model architecture type is concatenated into a reference quantity vector according to the model architecture type dimension, and the reference quantity vector can be represented as [n1*t1, n2*t2, …, n c *t c , …, n C *t C ], and correspondingly, the target matching vector determined subsequently is also calculated by similarity calculation using the above reference quantity vector.

[0079] In this embodiment, the average processing duration and the historical request quantity are multiplied to obtain the reference quantity vector, and the calculation resource quantity corresponding to different model architecture types is represented by the processing duration, thereby improving the rationality of the traffic matching, and further improving the efficiency of model invocation.

[0080] Optionally, determining the target matching vector most similar to the reference quantity vector from the preset traffic matching vectors comprises:

[0081] For any traffic matching vector, the preset coefficient is initialized to a preset value, the traffic matching vector is multiplied by the preset coefficient, the multiplication result is subtracted from the reference quantity vector element by element to obtain a subtraction result, and the sum of the absolute values of all subtraction results is determined as a difference value;

[0082] The preset coefficient is updated according to the preset step, and the step of multiplying the traffic matching vector by the preset coefficient is performed again until the subtraction result calculated according to the updated preset coefficient is greater than the subtraction result calculated according to the updated preset coefficient, and the subtraction result calculated according to the updated preset coefficient is determined as the optimization parameter.

[0083] Compare the optimization parameters corresponding to all flow ratio vectors, and determine the flow ratio vector with the smallest optimization parameters as the target flow ratio vector.

[0084] The preset coefficient can refer to the scaling factor of the flow ratio vector, and the preset value can be 1. The preset step size can refer to the adjustment amount when adjusting the preset coefficient. In this embodiment, the preset step size can be 0.5.

[0085] Specifically, assuming the preset coefficient is Q, the product of the flow ratio vector and the preset coefficient can be expressed as: The result of the multiplication is subtracted element-by-element from the reference vector, and the sum of the absolute values ​​of all subtractions is taken as the difference value, denoted as β. If the initial value of the preset coefficient is Q1, then the difference value calculated using the initial value can be expressed as β1.

[0086] The preset coefficients are updated according to a preset step size of l. The preset coefficients are adjusted by addition, i.e., by assigning a value to the preset coefficient Q using the formula Q = Q + l. The difference value β2 is then recalculated. β1 and β2 are compared. If β1 is greater than β2, it means that the adjusted preset coefficients make the flow ratio vector closer to the reference quantity vector after scaling. Continuing to adjust the preset coefficients may make the flow ratio vector even closer to the reference quantity vector after scaling. Therefore, the preset coefficients are updated again according to the preset step size, and the preset coefficient Q is assigned a value using the formula Q = Q + l. The difference value β3 is then recalculated. This process is repeated until the v-th difference value is less than the (v+1)-th difference value. This indicates that adjusting the preset coefficients at this point will only make the flow ratio vector move further away from the reference quantity vector after scaling. The v-th difference value is then determined as the optimization parameter.

[0087] The optimization parameters corresponding to all flow ratio vectors are compared. The flow ratio vector with the smallest optimization parameter is the flow ratio method that best matches the reference quantity vector. Therefore, the flow ratio vector with the smallest optimization parameter is taken as the target flow ratio vector.

[0088] In this embodiment, a preset coefficient is used as the scaling coefficient to iteratively search for the optimal difference value, thereby determining the target matching vector. Compared with conventional similarity calculation methods, this method can more accurately determine the traffic matching vector that is most similar to the reference quantity vector, thereby enabling more accurate load balancing on the server side and further improving the efficiency of model invocation.

[0089] Optionally, sending the request data to the target channel includes:

[0090] The decoded request data is re-encoded according to the encoding method of the corresponding target decoding method to obtain re-encoded data;

[0091] The recoded data is sent to the target channel.

[0092] The re-encoded data can be data obtained by re-encoding the decoded request data.

[0093] Specifically, in the codec, the decoding mode usually corresponds to the encoding mode one by one, and in the case of a known target decoding mode, the encoding mode corresponding to the target decoding mode is known by default.

[0094] Since the server needs to face multiple callers, and the storage space of the server is limited, after decoding the encoded request data, in order to save storage space, the encoded request data is not retained, but in the transmission process of the target channel, the encoded form of the request data needs to be transmitted to ensure transmission efficiency, therefore, the decoded request data is re-encoded.

[0095] In this embodiment, the re-encoding method is used to obtain re-encoded data, so that the storage space of the server is not occupied, the ability of the server to process parallel model calling requests is improved, and the efficiency of model calling is improved.

[0096] The above step of determining the target channel from the preset memory mapping according to the model name, and sending the request data to the target channel, connects the target model and the caller through the target channel, and the caller only needs to send the request to the receiving port of the server when sending the request, and then the server establishes the target channel for the caller and the target model. When the model in the server changes or updates, the caller does not need to change the sending address, which saves the communication cost between the caller and the model maintainer, and at the same time, it is convenient for the subsequent model maintainer to dynamically forward and expand or shrink the model service, thereby improving the efficiency of model calling.

[0097] In step S204, the request data is input into the loaded target model for calculation, and the calculation result is returned to the caller through the target channel.

[0098] The loaded target model is used to parse the encoded request data to obtain the to-be-processed data for model calculation, and the loaded target model takes the to-be-processed data as input data for model inference to obtain the final output, that is, the calculation result.

[0099] Specifically, after the caller receives the calculation result, the target channel is initialized, that is, the connection relationship between the target model port and the caller port is deleted, and the server waits to receive the calling request sent by other callers. At this time, the target model port can be used to distribute the model calling tasks of other callers.

[0100] The step of inputting the request data into the loaded target model for calculation and returning the calculation result to the calling party through the target channel, processes the request data through the loaded target model in the server, and sends the processing result to the calling party through the target channel, thereby realizing non-local calling of the complex model.

[0101] The embodiment parses the request website to obtain the demand model type, determines the corresponding target structure definition to decode the request data, obtains the model name, and then determines the target model and the target channel of the calling party according to the model name, so that the corresponding target model can be directly determined according to the model name, the model inadaptation condition is avoided, the model calling efficiency is improved, the calling party and the model are completely decoupled, when the model changes, the target channel can still be allocated through the model name, the communication and coordination cost of the model maintenance party and the calling party is saved, and the model calling efficiency is improved.

[0102] Referring to Figure 3 The model calling method provided in the embodiment two is a flowchart of a model calling method, after the model calling service of the calling party is completed, the connection relationship between the calling party and the target model can be directly deleted, that is, the target channel is initialized, or the target channel can be reserved according to the receiving time interval of the request data.

[0103] After the model calling service of the calling party is completed, the process of initializing the target channel is described in the embodiment one, and will not be repeated here.

[0104] After the model calling service of the calling party is completed, the process of reserving the target channel according to the receiving time interval of the request data includes:

[0105] In step S301, the request data sequence of the calling party is obtained, and the time interval average of adjacent request data in the request data sequence is calculated.

[0106] In step S302, when the time interval average is less than a preset time threshold, the target channel is reserved.

[0107] In step S303, when the second request website and the encoded second request data sent by the calling party are received, and the second name obtained by decoding the encoded second request data is consistent with the model name, the second request data is directly sent to the target channel.

[0108] The request data sequence can be all calling requests and corresponding sending time stamps sent by the calling party in a target time period, and the time interval average can be the average of the sending time stamp intervals of all adjacent request data.

[0109] In the embodiment, the preset time threshold can be set as 30 minutes, and the second request website and the encoded second request data can be the new model calling request content sent by the invoker.

[0110] Specifically, when the time interval mean is less than the preset time threshold, it indicates that the invoker needs to frequently use the model calling service, and at this time, the processing of initializing the target channel after each model calling service will cause more additional channel connection overheads, therefore, the target channel, i.e., the port of the target server, is directly kept connected with the invoker.

[0111] It should be noted that after the target channel is kept, if the model calling request of the invoker is not received within the preset time threshold, the target channel is initialized to avoid resource occupation.

[0112] The embodiment keeps the target channel according to the receiving time interval of the request data, avoids the overheads of newly building a connection for the subsequent request of the same invoker, and improves the efficiency of model calling.

[0113] corresponding to the model calling method of the above embodiment, Figure 4 a structural block diagram of a model calling device provided by Embodiment Three of the application is shown, the model calling device is applied to a server, and the server is connected with a computer device corresponding to a client to obtain a request website and encoded request data sent by an invoker, the invoker can be a user of the client, and the server is internally deployed with at least one loaded model to provide a model calling service for the invoker. For convenience of description, only parts related to the embodiments of the application are shown.

[0114] Referring to Figure 4 The model calling device comprises:

[0115] A type determination module 41 is configured to obtain the request website and the encoded request data of the invoker, perform string comparison between the request website and a preset type string, and if the comparison result is consistent, determine that the type string is a demand model type.

[0116] A data decoding module 42 is configured to determine a target decoding mode corresponding to the demand model type from a preset decoding mode database according to the demand model type, decode the encoded request data according to the target decoding mode, and obtain a model name.

[0117] A channel determination module 43 is configured to determine a target channel from a preset memory mapping according to the model name, and send the request data to the target channel, the memory mapping comprises a corresponding relationship between the model name and the channel, and the target channel is used to connect the invoker and the loaded target model.

[0118] The model calling module 44 is configured to input the request data into the loaded target model for calculation, and return the calculation result to the calling party through the target channel.

[0119] Optionally, the demand model type includes at least two model architecture types, and one model architecture type corresponds to one channel in the memory mapping.

[0120] The model calling device further includes:

[0121] The request statistics module is configured to count the demand model types of the at least two historical request websites, and obtain the historical request quantity of each model architecture type according to the counting result.

[0122] The traffic proportioning module is configured to splice the historical request quantity of each model architecture type into a reference quantity vector according to the model architecture type dimension, determine a target proportioning vector most similar to the reference quantity vector from a preset traffic proportioning vector, and proportion the traffic of all channels according to the target proportioning vector, so that the traffic of the target channel meets the calculation demand of the request data.

[0123] Optionally, the traffic proportioning module includes:

[0124] The time length correction unit is configured to obtain the average processing time length of each model architecture type, multiply the historical request quantity of the corresponding model architecture type by the average processing time length of the corresponding model architecture type to obtain a multiplication result of the corresponding model architecture type.

[0125] The vector splicing unit is configured to splice the multiplication result of each model architecture type into the reference quantity vector according to the model architecture type dimension.

[0126] Optionally, the traffic proportioning module includes:

[0127] The difference calculation unit is configured to multiply the preset coefficient by the traffic proportioning vector, subtract the multiplication result from the reference quantity vector element by element, and determine the sum of the absolute values of all subtraction results as a difference value.

[0128] The coefficient updating unit is configured to update the preset coefficient by a preset step, and perform again the step of multiplying the traffic proportioning vector by the preset coefficient until the difference value calculated according to the updated preset coefficient is greater than the difference value calculated according to the un-updated preset coefficient, and determine the difference value calculated according to the un-updated preset coefficient as an optimization parameter.

[0129] The vector determination unit is configured to compare the optimization parameters corresponding to all traffic proportioning vectors, and determine the traffic proportioning vector with the minimum optimization parameter as the target proportioning vector.

[0130] Optionally, the model calling apparatus further comprises:

[0131] a mark comparison module, configured to, if the comparison result is inconsistent, perform string comparison between the preset mark string and the request website to obtain a position of the mark string in the request website;

[0132] a string updating module, configured to extract the to-be-updated string from the request website according to the position and the preset separator, and update the to-be-updated string to the type string.

[0133] Optionally, the string updating module comprises:

[0134] a data decoding unit, configured to determine an updating decoding mode corresponding to the to-be-updated string from the decoding mode set, decode the request data according to the updating decoding mode, and obtain the model name;

[0135] a memory detection unit, configured to, when detecting that the model name is contained in the memory mapping, update the to-be-updated string to the type string.

[0136] Optionally, the channel determination module 43 comprises:

[0137] a re-encoding unit, configured to re-encode the decoded request data according to an encoding mode corresponding to the target decoding mode to obtain re-encoded data;

[0138] a data sending unit, configured to send the re-encoded data to the target channel.

[0139] Optionally, the model calling apparatus further comprises:

[0140] a sequence analysis module, configured to obtain a request data sequence of the calling party, and calculate a time interval average of adjacent request data in the request data sequence;

[0141] a channel reservation module, configured to, when the time interval average is less than a preset time threshold, reserve the target channel;

[0142] a channel sending module, configured to, when receiving the second request website and the encoded second request data sent by the calling party, and the second name obtained by decoding the encoded second request data is consistent with the model name, directly send the second request data to the target channel.

[0143] It should be noted that the information interaction, execution process and the like between the above modules and units are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, which will not be described here.

[0144] Figure 5This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executing the computer program, implements the steps in any of the above-described model invocation method embodiments.

[0145] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0146] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0147] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0148] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above device can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and when the processor executes the computer program, the steps of the above-mentioned method embodiment can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0149] The present application realizes all or part of the processes in the above-mentioned embodiment methods, which can also be completed by a computer program product. When the computer program product runs on the computer device, it makes the computer device execute the steps that can realize the above-mentioned method embodiments.

[0150] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0151] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0152] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other manners. For example, the described apparatus / computer device embodiments are merely schematic. For example, the division of the modules or units can be different, and each can include a plurality of sub-units. Some or all of the modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0153] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0154] The above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A model invocation method, characterized in that, The method includes: Obtain the request URL and encoded request data from the caller, compare the request URL with a preset type string, and if the comparison results match, determine that the type string is the requirement model type; Based on the required model type, a target decoding method corresponding to the required model type is determined from a preset set of decoding methods. The encoded request data is then decoded according to the target decoding method to obtain the model name. The target channel is determined from the preset memory mapping based on the model name, and the request data is sent to the target channel. The memory mapping includes the correspondence between model names and channels, and the target channel is used to connect the caller with the loaded target model. The requested data is input into the loaded target model for calculation, and the calculation result is returned to the caller through the target channel. The requirement model type includes at least two model architecture types, and one model architecture type corresponds to one channel in the memory mapping. Before determining the target channel from the preset memory mapping based on the model name, the method further includes: Statistical analysis is performed on the demand model types of at least two historical request URLs obtained, and the number of historical requests for each model architecture type is obtained based on the statistical results; The historical request counts for each model architecture type are concatenated into a reference count vector according to the model architecture type dimension. A target allocation vector that is most similar to the reference count vector is determined from the preset traffic allocation vector. Traffic allocation is performed on all channels according to the target allocation vector. The traffic allocation is used to ensure that the traffic of the target channel meets the calculation requirements of the request data. After concatenating all historical request counts into a reference count vector according to the dimension of model architecture type, the following is included: Obtain the average processing time for each model architecture type. For any model architecture type, multiply the historical request count corresponding to that model architecture type by the average processing time corresponding to that model architecture type to obtain the multiplication result corresponding to that model architecture type. The multiplication results of each model architecture type are concatenated into the reference quantity vector according to the dimensions of the model architecture type.

2. The model invocation method according to claim 1, characterized in that, The step of determining the target allocation vector that is most similar to the reference quantity vector from the preset flow allocation vector includes: For any flow ratio vector, initialize the preset coefficient to a preset value, multiply the flow ratio vector by the preset coefficient, subtract the result of the multiplication from the reference quantity vector element by element, and determine the sum of the absolute values ​​of all the subtraction results as the difference value. The preset coefficients are updated according to a preset step size, and the step of multiplying the flow ratio vector with the preset coefficients is performed again until the difference value calculated based on the updated preset coefficients is greater than the difference value calculated based on the preset coefficients before the update. The difference value calculated based on the preset coefficients before the update is then determined to be the optimization parameter. Compare the optimization parameters corresponding to all flow ratio vectors, and determine the flow ratio vector with the smallest optimization parameters as the target flow ratio vector.

3. The model invocation method according to claim 1, characterized in that, After comparing the requested URL with a preset type string, the method further includes: If the comparison results are inconsistent, a preset identifier string is used to compare the string with the request URL to obtain the position of the identifier string in the request URL; Based on the location and the preset delimiter, the string to be updated is extracted from the request URL. The update decoding method corresponding to the string to be updated is determined from the decoding method set. The request data is decoded according to the update decoding method to obtain the model name. When the memory mapping is detected to contain the model name, the string to be updated is updated to the type string.

4. The model invocation method according to claim 1, characterized in that, Sending the request data to the target channel includes: The decoded request data is re-encoded according to the encoding method corresponding to the target decoding method to obtain re-encoded data; The recoded data is sent to the target channel.

5. The model invocation method according to any one of claims 1 to 4, characterized in that, After sending the request data to the target channel, the method further includes: Obtain the request data sequence from the caller, and calculate the average time interval between adjacent request data in the request data sequence; When the average time interval is less than a preset time threshold, the target channel is retained; Upon receiving the second request URL and encoded second request data sent by the caller, and when the second name obtained by decoding the encoded second request data matches the model name, the second request data is directly sent to the target channel.

6. A model recall device, characterized in that, The model invocation device includes: The type determination module is used to obtain the request URL and encoded request data from the caller, compare the request URL with a preset type string, and if the comparison result is consistent, determine that the type string is the requirement model type. The data decoding module is used to determine the target decoding method corresponding to the demand model type from a preset decoding method database according to the demand model type, and decode the encoded request data according to the target decoding method to obtain the model name; The channel determination module is used to determine the target channel from the preset memory mapping according to the model name, and send the request data to the target channel. The memory mapping includes the correspondence between model names and channels, and the target channel is used to connect the caller and the loaded target model. The model invocation module is used to input the requested data into the loaded target model for calculation, and return the calculation result to the invoker through the target channel; The requirement model type includes at least two model architecture types, each model architecture type corresponding to a channel in memory mapping. The model invocation device further includes: The request statistics module is used to count the demand model types of at least two historical request URLs obtained, and to obtain the number of historical requests for each model architecture type based on the statistical results. The traffic allocation module is used to concatenate the historical request counts of each model architecture type into a reference count vector according to the model architecture type dimension, determine the target allocation vector that is most similar to the reference count vector from the preset traffic allocation vector, and perform traffic allocation on all channels according to the target allocation vector. The traffic allocation is used to ensure that the traffic of the target channel meets the calculation requirements of the request data. The flow rate matching module includes: The duration correction unit is used to obtain the average processing time for each model architecture type. For any model architecture type, the number of historical requests corresponding to the model architecture type is multiplied by the average processing time corresponding to the model architecture type to obtain the multiplication result corresponding to the model architecture type. The vector concatenation unit is used to concatenate the multiplication results of each model architecture type into the reference quantity vector according to the dimension of the model architecture type.

7. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the model invocation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model invocation method as described in any one of claims 1 to 5.

Images