Large language model analysis method and device and related equipment

CN118916042APending Publication Date: 2024-11-08DATACANVAS LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410954116.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08

Smart Images

  • Figure CN118916042A_ABST
    Figure CN118916042A_ABST
Patent Text Reader

Abstract

The invention provides a big language model analysis method and device and related equipment, and relates to the technical field of big language models.The method comprises the steps that S1, big language model information is obtained, and the big language model information comprises a model file of a target model and frame data, the frame data indicates a target application frame corresponding to the target model; s2, determining a target mirror image file corresponding to the target application framework according to the framework data; s3, starting a container according to the target mirror image file, and running a preset model analysis service in the container; and S4, under the condition that the model file is loaded into the container and the model analysis service successfully analyzes the target model, model analysis information output by the model analysis service is obtained, and the model analysis information comprises model input data and model output data of the target model. The method can improve the analysis efficiency of the user on the external model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of large language models, and in particular to a large language model analysis method, apparatus, and related equipment. Background Art

[0002] In the process of production and operation, enterprises can better deal with various complex production activities and business processes through the use of large-scale language models (LLM).

[0003] At present, in addition to obtaining large language models through self-development, enterprises can also obtain large language models through open source technology communities and third-party professional technology service companies. Such non-self-developed large language models are generally called "external large language models."

[0004] For the above-mentioned "external model", before connecting it to the company's business system, the corresponding large language model analysis work needs to be completed to determine whether the "external large language model" can work normally and whether it meets business needs. The above analysis work requires the construction of the corresponding large language model analysis environment, and the construction of the large language model analysis environment is highly dependent on manual work, which makes the construction efficiency of the large language model analysis environment relatively low, which in turn causes the analysis efficiency of the external large language model to be relatively low. Summary of the invention

[0005] The purpose of the present disclosure is to provide a large language model analysis method, apparatus and related equipment to solve the technical problem of low analysis efficiency of external large language models in related technologies.

[0006] In a first aspect, an embodiment of the present disclosure provides a large language model analysis method, including:

[0007] Step S1: Acquire model information, wherein the model information includes a model file of a target model and framework data, wherein the framework data indicates a target application framework corresponding to the target model;

[0008] Step S2: determining the target image file corresponding to the target application framework according to the framework data;

[0009] Step S3: starting a container according to the target image file, and running a preset model analysis service in the container;

[0010] Step S4: When the model file is loaded into the container and the model analysis service successfully analyzes the target model, the model analysis information output by the model analysis service is obtained, and the model analysis information includes the model input data and the model output data of the target model.

[0011] In one embodiment, the step S2: determining the target image file corresponding to the target application framework according to the framework data includes:

[0012] Step S201: obtaining a plurality of candidate image files and a plurality of candidate algorithm packages, each of the candidate image files corresponding to an application framework, and among the plurality of candidate image files, different candidate image files correspond to different application frameworks, each of the candidate algorithm packages corresponds to an image file, and the image files corresponding to the plurality of candidate algorithm packages are different;

[0013] Step S202: when the plurality of candidate image files include a candidate image file matching the framework data, determining the candidate image file matching the framework data as the target image file;

[0014] Step S203: when the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages include a candidate algorithm package matching the framework data, determining the candidate algorithm package matching the framework data as a first target algorithm package, and generating the target image file according to the first target algorithm package;

[0015] Step S204: When the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages do not include a candidate algorithm package matching the framework data, receiving a second target algorithm package uploaded by a user, and generating the target image file according to the second target algorithm package.

[0016] In one embodiment, the framework data includes first sub-data indicating a framework category of the target application framework, and second sub-data indicating a framework version of the target application framework;

[0017] Each of the candidate image files corresponds to a framework identifier, and the framework identifier is determined based on the framework category and framework version of the corresponding application framework;

[0018] When the first candidate image file meets the following conditions, the first candidate image file is determined to be a candidate image file matching the framework data:

[0019] The framework category indicated by the framework identifier of the candidate image file matches the first sub-data, and the framework version indicated by the framework identifier corresponding to the candidate image file matches the second sub-data, wherein the first candidate image file is one of the multiple candidate image files.

[0020] In one embodiment, the large language model information further includes: first reference data indicating the format of the model file, and second reference data indicating the size of the model file;

[0021] The step S3: starting the container according to the target image file comprises:

[0022] Step S301: determining computing resources according to the first reference data, the second reference data and the framework category of the target application framework;

[0023] Step S302: starting the container according to the computing resources and the target image file.

[0024] In one embodiment, after the step S4: obtaining the model analysis information output by the model analysis service, the method further includes:

[0025] Step S5: generating target sample data according to the model analysis information, wherein the target sample data includes sample input sub-data and sample output sub-data;

[0026] Step S6: inputting the sample input sub-data into the target model to obtain a sample output result;

[0027] Step S7: when the sample output result is within the value range indicated by the sample output sub-data, generate first test information, wherein the first test information indicates that the target model is an available model.

[0028] In one embodiment, the step S5: generating corresponding target sample data according to the model analysis information includes:

[0029] Step S501: generating candidate sample data according to the model analysis information, wherein the candidate sample data includes candidate input sub-data and candidate output sub-data;

[0030] Step S502: when the sample update information is obtained, at least one of the candidate input sub-data and the candidate output sub-data is updated according to the sample update information to obtain the target sample data.

[0031] In one embodiment, after the step S3: starting a container according to the target image file and running a preset model analysis service in the container, the method further includes:

[0032] Step S8: loading the model file into the container, and generating an analysis failure log when the model analysis service fails to analyze the target model;

[0033] Step S9: Output the analysis failure log.

[0034] In a second aspect, the present disclosure provides a large language model analysis device, including:

[0035] An acquisition module, configured to acquire large language model information, wherein the large language model information includes a model file of a target model and framework data, wherein the framework data indicates a target application framework corresponding to the target model;

[0036] A determination module, used to determine a target image file corresponding to the target application framework according to the framework data;

[0037] A startup module, used to start a container according to the target image file and run a preset model analysis service in the container;

[0038] The analysis module is used to obtain model analysis information output by the model analysis service when the model file is loaded into the container and the model analysis service successfully analyzes the target model, wherein the model analysis information includes model input data and model output data of the target model.

[0039] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned large language model analysis method when executed by the processor.

[0040] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned large language model analysis method are implemented.

[0041] In a fifth aspect, the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the above-mentioned large language model analysis method.

[0042] In the disclosed embodiment, the target image file is determined based on the framework data included in the large language model information, and then the container is started based on the target image file, and the model analysis service is run in the container, so as to replace manual operation by automatically configuring the model analysis environment corresponding to the target model, avoid the situation where the user manually builds the model analysis environment, reduce the difficulty of using the model analysis function, and avoid the constraints caused by human factors, thereby improving the user's analysis efficiency of the external large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of a large language model analysis method provided by an embodiment of the present disclosure;

[0044] Figure 2 It is a model service architecture diagram of a cloud platform provided by an embodiment of the present disclosure;

[0045] Figure 3 It is a schematic diagram of an image building process provided by an embodiment of the present disclosure;

[0046] Figure 4 is a schematic diagram of a recommended process of mirroring and computing resources provided by an embodiment of the present disclosure;

[0047] Figure 5 is a schematic diagram of a sample data and SDK generation process provided by an embodiment of the present disclosure;

[0048] Figure 6 is a schematic diagram of a large language model analysis device provided by an embodiment of the present disclosure;

[0049] Figure 7 It is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0051] The following is an explanation of the various names involved in the embodiments of the present disclosure.

[0052] Large language model: refers to a large language model (LLM), which is a language model with a large parameter scale. It is designed to understand and generate human language. It is trained with a large amount of text data and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc.

[0053] Model dissection: read model files and parse model information.

[0054] Framework: A library, interface, or tool used to support the dissection of machine learning models.

[0055] Container: A running instance created based on an image file, which can be understood as a virtualized operating system that includes a specific framework.

[0056] Image file: A type of file that corresponds to a specific framework and is used to start a container.

[0057] Algorithm package: refers to a set of software libraries or tool collections that implement specific algorithms or methods and are used to build image files.

[0058] In the related art, for a large language model obtained from an external channel, before connecting it to a business system, the user needs to select a specific image file and start a container that is adapted to the framework operating environment of the large language model based on the image file. Then, the corresponding framework is run in the container and the construction of the large language model is completed, so as to determine whether the large language model can work normally and whether it meets the business needs. This type of manual operation not only requires the user to have certain environmental configuration knowledge, but also takes a lot of time in the configuration stage of the model operating environment. For example, if the user is not familiar with the large language model obtained from the external channel, the user will repeatedly debug the model operating environment, which seriously affects the user's analysis efficiency of the large language model obtained from the external channel.

[0059] To solve the above problems, the present disclosure provides a large language model analysis method, such as Figure 1 As shown, the large language model analysis method includes:

[0060] Step S1: Obtain large language model information.

[0061] The large language model information includes a model file of a target model and framework data, wherein the framework data indicates a target application framework corresponding to the target model.

[0062] The above large language model information is provided by the user, and the above model file is a type of file used to create a target model in a container adapted to the target model (referring to a container including a framework supporting the target model).

[0063] Step S2: determining a target image file corresponding to the target application framework according to the framework data.

[0064] Exemplarily, an image file warehouse storing a plurality of pre-set image files may be maintained, and after receiving and parsing the framework data included in the large language model information uploaded by the user, the target image file may be determined from the plurality of image files in the image file warehouse based on the framework data, wherein the plurality of pre-set image files may be the plurality of image files that are most frequently used by users among all image files.

[0065] Step S3: Start a container according to the target image file, and run a preset model analysis service in the container.

[0066] Wherein, the above container includes the target application framework.

[0067] Step S4: when the model file is loaded into the container and the model analysis service successfully analyzes the target model, the model analysis information output by the model analysis service is obtained.

[0068] The model analysis information includes model input data and model output data of the target model.

[0069] The model input data indicates the number of input parameters of the target model, the parameter value type (such as floating point type, enumeration type, etc.), the parameter value range, the corresponding business meaning of the input parameters, etc.

[0070] The model output data indicates the number of parameters, parameter value types, parameter value ranges, business meanings corresponding to the output parameters, etc. of the target model's output parameters.

[0071] In some implementations, the model analysis information may also include model structure data and model weight data of the target model.

[0072] Among them, the model structure data indicates the ways in which the various elements of the target model are interrelated and interact with each other, including the quantity ratio, arrangement order, combination method, etc. of the constituent elements, such as the network architecture, convolution depth, convolution method, pooling method, etc. of the target model.

[0073] The model weight data indicates the element sizes of various elements of the target model, such as the number of input channels, the number of output channels, the convolution kernel size, the bias, etc.

[0074] In the present disclosure, the target image file is determined according to the framework data included in the large language model information, and then the container is started based on the target image file, and the model analysis service is run in the container, so as to replace manual operation by automatically configuring the model analysis environment corresponding to the target model, avoid the situation where users manually build the model analysis environment, reduce the difficulty of using the model analysis function, and avoid the constraints caused by human factors, thereby improving the user's analysis efficiency of the external large language model.

[0075] In one embodiment, the step S2: determining the target image file corresponding to the target application framework according to the framework data includes:

[0076] Step S201: obtaining a plurality of candidate image files and a plurality of candidate algorithm packages, each of the candidate image files corresponding to an application framework, and among the plurality of candidate image files, different candidate image files correspond to different application frameworks, each of the candidate algorithm packages corresponds to an image file, and the image files corresponding to the plurality of candidate algorithm packages are different;

[0077] Step S202: when the plurality of candidate image files include a candidate image file matching the framework data, determining the candidate image file matching the framework data as the target image file;

[0078] Step S203: when the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages include a candidate algorithm package matching the framework data, determining the candidate algorithm package matching the framework data as a first target algorithm package, and generating the target image file according to the first target algorithm package;

[0079] Step S204: When the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages do not include a candidate algorithm package matching the framework data, receiving a second target algorithm package uploaded by a user, and generating the target image file according to the second target algorithm package.

[0080] Exemplarily, when the method disclosed in the present invention is applied on a cloud platform, the algorithm package provided by the cloud platform and the algorithm package provided by the user accessing the cloud platform can be integrated to form the aforementioned multiple candidate algorithm packages; then, some candidate algorithm packages that match user needs (such as those with high user usage frequency or adapted to the business needs of the service rented by the user) are selected from the multiple candidate algorithm packages, and image files are constructed accordingly to form the aforementioned multiple candidate image files, thereby reducing the storage resource occupancy rate of the multiple candidate image files in the user's private space, wherein the multiple candidate algorithm packages are stored in the cloud platform, and the multiple candidate image files are stored in the local storage of the user's corresponding private space;

[0081] After obtaining the above-mentioned framework data, a preliminary search is first performed on multiple candidate image files stored in the private space corresponding to the user based on the framework data. If the search fails, a query request carrying the framework data is sent to the cloud platform to further search among multiple candidate algorithm packages stored in the cloud platform based on the framework data. Through this setting, while ensuring the efficiency of obtaining the target image file, the storage occupancy rate of the user's private space is reduced as much as possible, avoiding freezes due to excessive storage occupancy of the user's private space, and optimizing the user's experience of using the private space allocated by the cloud platform.

[0082] Among them, when multiple candidate image files and multiple candidate algorithm packages cannot match the framework data, users are supported to upload the second target algorithm package by themselves to form a target image file, which can ensure that in extreme cases, the method described in the present disclosure can still be successfully executed, thereby improving the versatility of the method described in the present disclosure; and after generating the target image file based on the second target algorithm package, the second target algorithm package can also be added to the multiple candidate algorithm packages, and the target image file can be added to the multiple candidate image files to enrich the number of candidate algorithm packages and candidate image files, while facilitating users to reuse the second target algorithm package and the target image file.

[0083] Exemplarily, the process of building a target image file based on the first target algorithm package or the second target algorithm package can be: building a dockerfile file according to the first target algorithm package or the second target algorithm package, and starting the docker tool according to the dockerfile file to complete the construction of the target image file.

[0084] In one embodiment, the framework data includes first sub-data indicating a framework category of the target application framework, and second sub-data indicating a framework version of the target application framework;

[0085] Each of the candidate image files corresponds to a framework identifier, and the framework identifier is determined based on the framework category and framework version of the corresponding application framework;

[0086] When the first candidate image file meets the following conditions, the first candidate image file is determined to be a candidate image file matching the framework data:

[0087] The framework category indicated by the framework identifier of the candidate image file matches the first sub-data, and the framework version indicated by the framework identifier corresponding to the candidate image file matches the second sub-data, wherein the first candidate image file is one of the multiple candidate image files.

[0088] Similarly, the first target algorithm package meets the following conditions:

[0089] The framework category indicated by the framework identifier of the image file corresponding to the first target algorithm package matches the first sub-data, and the framework version indicated by the framework identifier of the image file corresponding to the first target algorithm package matches the second sub-data.

[0090] In this embodiment, a framework is uniquely identified based on the framework category and framework version, and then the association between the framework, image file and algorithm package is completed to adapt to the feature differences between different frameworks and the feature differences between different versions of the same framework, making the application of the method described in the present disclosure more flexible and making the model analysis information output by applying the method described in the present disclosure more reliable.

[0091] For example, the framework categories include: jinja2, torch, transformers, etc.

[0092] In one embodiment, the large language model information further includes: first reference data indicating the format of the model file, and second reference data indicating the size of the model file;

[0093] The step S3: starting the container according to the target image file comprises:

[0094] Step S301: determining computing resources according to the first reference data, the second reference data and the framework category of the target application framework;

[0095] Step S302: starting the container according to the computing resources and the target image file.

[0096] In this embodiment, before starting a container according to the target image file, the minimum computing resource requirements for at least running the model file are determined based on the format of the model file, the size of the model file, and the framework category of the target application framework, and the container is started accordingly. On the premise of ensuring the normal startup of the container and the normal operation of the model file, the computing resources occupied by the container are minimized, thereby optimizing the overall computing resource utilization in the cloud platform.

[0097] For example, the computing resources include but are not limited to memory resources and CPU resources.

[0098] In one embodiment, after the step S4: obtaining the model analysis information output by the model analysis service, the method further includes:

[0099] Step S5: generating target sample data according to the model analysis information, wherein the target sample data includes sample input sub-data and sample output sub-data;

[0100] Step S6: inputting the sample input sub-data into the target model to obtain a sample output result;

[0101] Step S7: when the sample output result is within the value range indicated by the sample output sub-data, generate first test information, wherein the first test information indicates that the target model is an available model.

[0102] In this embodiment, after performing model analysis on the target model, the target model is further verified for availability based on the analysis results (referring to model analysis information) to determine whether the target model can work normally. This can further facilitate the user's testing work before applying the target model and enhance the user's experience.

[0103] For example, if the target model is used to predict weather conditions, the values ​​indicated by the sample output sub-data include "temperature rise" and "temperature drop"; after the sample input sub-data is input into the target model, if the sample output result is "temperature rise" or "temperature drop", the target model can be determined to be an available model; otherwise, when the sample output result is other than "temperature rise" and "temperature drop", the target model is determined to be an unavailable model.

[0104] In one embodiment, the step S5: generating corresponding target sample data according to the model analysis information includes:

[0105] Step S501: generating candidate sample data according to the model analysis information, wherein the candidate sample data includes candidate input sub-data and candidate output sub-data;

[0106] Step S502: when the sample update information is obtained, at least one of the candidate input sub-data and the candidate output sub-data is updated according to the sample update information to obtain the target sample data.

[0107] Among them, the alternative input sub-data can be obtained according to the aforementioned model input information, and the alternative output sub-data can be obtained according to the aforementioned model output information. After the alternative sample data is obtained, by obtaining the sample update information provided by the user, at least one of the alternative input sub-data and the alternative output sub-data is updated according to the sample update information, so as to support the user's customized data setting for the usability test of the target model, so that the application of the method described in the present disclosure is more flexible.

[0108] In one embodiment, in step S3: after starting a container according to the target image file and running a preset model analysis service in the container, the method further includes:

[0109] Step S8: loading the model file into the container, and generating an analysis failure log when the model analysis service fails to analyze the target model;

[0110] Step S9: Output the analysis failure log.

[0111] In this embodiment, when the analysis of the target model fails, the error information in the analysis process is recorded and output by generating an analysis failure log to facilitate the user to troubleshoot the error and reduce the adverse effects of interference factors (such as missing model files, abnormal container environment, abnormal model analysis service, etc.) on the user's target model analysis process.

[0112] In some implementations, after step S4: obtaining the model analysis information output by the model analysis service, the method further includes:

[0113] Step S10: Generate an SDK development kit for the target model based on the model analysis information. The SDK development kit can be downloaded by users. After downloading, users can upload the SDK development kit in other environments to easily load the target model in the other environments (such as actual production environments, internal test environments, etc.).

[0114] For example, after the user completes the analysis of the target model, it can be a rest API service or gRPC service based on the http protocol, which uses the target model to respond to the call of the business system online; or the target model can be integrated into the data batch processing system to complete the prediction of large batches of data during the periodic batch processing of data; or the target model can be directly embedded in the application or terminal device of the business system, and the model's prediction of the data can be directly converted into the behavior within the application, so as to meet the business needs of real-time prediction of the business system.

[0115] For ease of understanding, the following examples are provided:

[0116] After users access the cloud platform, they can Figure 2 As shown in the figure, three parts are set up: model interaction service (model interaction, which can be understood as a web-based UI interaction page), service for building images (docker image manager), and image list (image store, which contains image files of the current mainstream training framework).

[0117] Among them, Figure 3As shown, there are multiple image files in the image list (which can be understood as the aforementioned multiple candidate image files), and an image file is uniquely identified based on the training framework and the corresponding version. After the user uploads the framework and version (referring to the aforementioned framework data) corresponding to the model to be dissected (which can be understood as the aforementioned target model) through the model-based interaction service, it is searched in the image list. If the search fails (referring to the framework and version corresponding to the model to be dissected are not in the image list), the service (docker image manager) for building images provided by the cloud platform can be used to connect to the algorithm package warehouse (image Store) of the cloud platform according to the training framework information filled in by the user (which can be understood as the aforementioned framework data). The algorithm package warehouse includes multiple algorithm packages (which can be understood as the aforementioned multiple candidate algorithm packages), and the algorithm package matching the training framework information is determined (which can be understood as the first target algorithm package), and the dockerfile file is constructed accordingly, and the docker tool is started to build the image (which can be understood as the target image file), and the constructed image is uploaded to the algorithm package warehouse for subsequent reuse;

[0118] Among them, if there is no algorithm package matching the training framework information in the algorithm package warehouse, the user is allowed to automatically upload the algorithm package matching the training framework information (which can be understood as the second target algorithm package). The subsequent image building process is the same as the above description and will not be repeated here.

[0119] like Figure 4 As shown, when the user needs to dissect an external model (which can be understood as a target model), the model interaction service can be called. The user uploads the model to be dissected to temporary storage through the model interaction service, and the user also inputs the training framework (which can be understood as the first sub-data) and version information (which can be understood as the second sub-data) corresponding to the model to be dissected. After completing the model reception, the model interaction service stores it in temporary storage, and according to the format of the model file, the size of the model file, the model training framework and version information, it infers which image needs to be used in the model dissection process and the required computing resources, which include: CPU, memory.

[0120] Among them, the training framework has its corresponding / supported image, from which the image required for model dissection can be inferred.

[0121] Then, if Figure 5As shown, according to the image required for inference and the recommended resources, the model dissection service starts the corresponding image and loads the uploaded model file into the container. After the model dissection service is started, it reads the model and starts parsing. After the parsing is completed, the dissected information (such as model structure, input, output, weight, etc.) is stored in the background storage (model store), and sample data (which can be understood as alternative sample data) and the corresponding SDK development package are generated based on the dissection information; if an error occurs during the parsing process, the error information during the parsing process must be recorded in detail (which can be understood as an analysis failure log) and stored synchronously so that the error information can be output to the user.

[0122] Finally, the model anatomical information, generated sample data and corresponding SDK development kit are fed back to the user through the model interaction service;

[0123] Among them, if the model dissection is successful, the user can choose whether to adjust the input sample data. If not adjusted, the generated sample data (which can be understood as alternative sample data) will be used by default, and the model will be tested for usability based on the sample data (which can be understood as target sample data). After the test is completed, the results will be presented to the user; if the model dissection fails, the error information generated by the dissection process will be presented to the user in detail.

[0124] See also Figure 6 , Figure 6 A large language model analysis device provided by an embodiment of the present disclosure is Figure 6 As shown, the large language model analysis device 600 includes:

[0125] An acquisition module 601 is used to acquire large language model information, wherein the large language model information includes a model file and framework data of a target model, wherein the framework data indicates a target application framework corresponding to the target model;

[0126] A determination module 602 is used to determine a target image file corresponding to the target application framework according to the framework data;

[0127] A startup module 603 is used to start a container according to the target image file and run a preset model analysis service in the container;

[0128] The analysis module 604 is used to obtain the model analysis information output by the model analysis service when the model file is loaded into the container and the model analysis service successfully analyzes the target model, and the model analysis information includes the model input data and model output data of the target model.

[0129] In one embodiment, the determining module 602 is specifically configured to:

[0130] Obtain multiple candidate image files and multiple candidate algorithm packages, each of the candidate image files corresponds to an application framework, and among the multiple candidate image files, different candidate image files correspond to different application frameworks, each of the candidate algorithm packages corresponds to an image file, and the image files corresponding to the multiple candidate algorithm packages are different;

[0131] In a case where the multiple candidate image files include a candidate image file matching the framework data, determining the candidate image file matching the framework data as the target image file;

[0132] In a case where the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages include a candidate algorithm package matching the framework data, determining the candidate algorithm package matching the framework data as a first target algorithm package, and generating the target image file according to the first target algorithm package;

[0133] When the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages do not include a candidate algorithm package matching the framework data, a second target algorithm package uploaded by a user is received, and the target image file is generated according to the second target algorithm package.

[0134] In one embodiment, the framework data includes first sub-data indicating a framework category of the target application framework, and second sub-data indicating a framework version of the target application framework;

[0135] Each of the candidate image files corresponds to a framework identifier, and the framework identifier is determined based on the framework category and framework version of the corresponding application framework;

[0136] When the first candidate image file meets the following conditions, the first candidate image file is determined to be a candidate image file matching the framework data:

[0137] The framework category indicated by the framework identifier of the candidate image file matches the first sub-data, and the framework version indicated by the framework identifier corresponding to the candidate image file matches the second sub-data, wherein the first candidate image file is one of the multiple candidate image files.

[0138] In one embodiment, the large language model information further includes: first reference data indicating the format of the model file, and second reference data indicating the size of the model file;

[0139] The startup module 603 is specifically used for:

[0140] Determining computing resources according to the first reference data, the second reference data, and the framework category of the target application framework;

[0141] The container is started according to the computing resources and the target image file.

[0142] In one embodiment, the apparatus 600 further includes:

[0143] A sample generation module, used to generate target sample data according to the model analysis information, wherein the target sample data includes sample input sub-data and sample output sub-data;

[0144] A sample test module, used for inputting the sample input sub-data into the target model to obtain a sample output result;

[0145] A result analysis module is used to generate first test information when the sample output result is within the value range indicated by the sample output sub-data, wherein the first test information indicates that the target model is an available model.

[0146] In one embodiment, the sample generation module is specifically used to:

[0147] Generate candidate sample data according to the model analysis information, wherein the candidate sample data includes candidate input sub-data and candidate output sub-data;

[0148] When the sample update information is obtained, at least one of the candidate input sub-data and the candidate output sub-data is updated according to the sample update information to obtain the target sample data.

[0149] In one embodiment, the apparatus 600 further includes:

[0150] A log module, for generating an analysis failure log when the model file is loaded into the container and when the model analysis service fails to analyze the target model;

[0151] The output module is used to output the analysis failure log.

[0152] The large language model analysis device 600 provided in the embodiment of the present disclosure can implement each process in the above-mentioned large language model analysis method embodiment, and will not be described again here to avoid repetition.

[0153] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0154] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0155] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0156] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0157] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the large language model analysis method. For example, in some embodiments, the large language model analysis method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the large language model analysis method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the large language model analysis method in any other appropriate manner (for example, by means of firmware).

[0158] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0160] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0163] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0164] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0165] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large language model analysis method, characterized in that: The method comprises: Step S1: Acquire large language model information, the large language model information including a model file and framework data of a target model, the framework data indicating a target application framework corresponding to the target model; Step S2: determining the target image file corresponding to the target application framework according to the framework data; Step S3: starting a container according to the target image file, and running a preset model analysis service in the container; Step S4: When the model file is loaded into the container and the model analysis service successfully analyzes the target model, the model analysis information output by the model analysis service is obtained, and the model analysis information includes the model input data and the model output data of the target model.

2. The method according to claim 1, characterized in that The step S2: determining the target image file corresponding to the target application framework according to the framework data, comprises: Step S201: obtaining a plurality of candidate image files and a plurality of candidate algorithm packages, each of the candidate image files corresponding to an application framework, and among the plurality of candidate image files, different candidate image files correspond to different application frameworks, each of the candidate algorithm packages corresponds to an image file, and the image files corresponding to the plurality of candidate algorithm packages are different; Step S202: when the plurality of candidate image files include a candidate image file matching the framework data, determining the candidate image file matching the framework data as the target image file; Step S203: when the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages include a candidate algorithm package matching the framework data, determining the candidate algorithm package matching the framework data as a first target algorithm package, and generating the target image file according to the first target algorithm package; Step S204: When the multiple candidate image files do not include a candidate image file matching the framework data, and the multiple candidate algorithm packages do not include a candidate algorithm package matching the framework data, receiving a second target algorithm package uploaded by a user, and generating the target image file according to the second target algorithm package.

3. The method according to claim 2, characterized in that The framework data includes first sub-data indicating a framework category of the target application framework, and second sub-data indicating a framework version of the target application framework; Each of the candidate image files corresponds to a framework identifier, and the framework identifier is determined based on the framework category and framework version of the corresponding application framework; When the first candidate image file meets the following conditions, the first candidate image file is determined to be a candidate image file matching the framework data: The framework category indicated by the framework identifier of the candidate image file matches the first sub-data, and the framework version indicated by the framework identifier corresponding to the candidate image file matches the second sub-data, wherein the first candidate image file is one of the multiple candidate image files.

4. The method according to claim 1, characterized in that: The large language model information further includes: first reference data indicating a format of the model file, and second reference data indicating a size of the model file; The step S3: starting the container according to the target image file comprises: Step S301: determining computing resources according to the first reference data, the second reference data and the framework category of the target application framework; Step S302: Start the container according to the computing resources and the target image file.

5. The method according to claim 1, characterized in that After the step S4: obtaining the model analysis information output by the model analysis service, the method further includes: Step S5: generating target sample data according to the model analysis information, wherein the target sample data includes sample input sub-data and sample output sub-data; Step S6: inputting the sample input sub-data into the target model to obtain a sample output result; Step S7: when the sample output result is within the value range indicated by the sample output sub-data, generate first test information, wherein the first test information indicates that the target model is an available model.

6. The method according to claim 5, characterized in that The step S5: generating corresponding target sample data according to the model analysis information includes: Step S501: generating candidate sample data according to the model analysis information, wherein the candidate sample data includes candidate input sub-data and candidate output sub-data; Step S502: when the sample update information is obtained, at least one of the candidate input sub-data and the candidate output sub-data is updated according to the sample update information to obtain the target sample data.

7. The method according to claim 1, characterized in that In step S3, after starting a container according to the target image file and running a preset model analysis service in the container, the method further includes: Step S8: loading the model file into the container, and generating an analysis failure log when the model analysis service fails to analyze the target model; Step S9: Output the analysis failure log.

8. A large language model analysis device, characterized in that: The device comprises: An acquisition module, configured to acquire large language model information, wherein the large language model information includes a model file of a target model and framework data, wherein the framework data indicates a target application framework corresponding to the target model; A determination module, used to determine a target image file corresponding to the target application framework according to the framework data; A startup module, used to start a container according to the target image file and run a preset model analysis service in the container; The analysis module is used to obtain model analysis information output by the model analysis service when the model file is loaded into the container and the model analysis service successfully analyzes the target model, wherein the model analysis information includes model input data and model output data of the target model.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Document generation method and device, equipment, storage medium and program product

    CN120429009A