Large language model (LLM)-based application identification method and apparatus

Abstract

Embodiments of the present disclosure provide a large language model (LLM)-based application identification method and apparatus. The method comprises: analyzing feature similarity between a plurality of pieces of application data by means of an LLM, and classifying the application data on the basis of the feature similarity; and training a learning model on the basis of the classified application data, so as to identify applications of a same category among a plurality of pieces of application data to be identified. The present disclosure solves the problems that existing models cannot be adaptive when facing application data changes, and when part of feature data of application data is changed due to application changes, the models cannot correctly identify applications and require a large amount of annotated data for training.

Description

Application identification method and device based on large language model LLM

[0001] Cross-reference to Related Applications

[0002] The present disclosure is based on Chinese Patent Application No. CN202411044379.6 entitled "Application identification method and device based on large language model LLM" filed on July 31, 2024, and claims priority to the patent application, the disclosure of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] Embodiments of the present disclosure relate to the field of network operation and network security, and in particular, to an application identification method and device based on a large language model (LLM). BACKGROUND

[0004] With the rapid development of Internet applications, the phenomenon of frequent updates and name changes, icon changes, and other identification changes of application programs (Apps) is becoming increasingly common. In particular, malicious applications are increasing, which disguise themselves in this way to evade security detection, causing serious security risks to users. In the prior art, there are the following limitations in dealing with application analysis and identification:

[0005] 1. High-quality data dependency: models usually need a large amount of high-quality labeled data for training.

[0006] 2. Dynamic adaptability: application data is dynamically changing, and once the model is trained, its ability to adapt to new changes is limited, and it needs to be retrained regularly to maintain its effectiveness. SUMMARY

[0007] Embodiments of the present disclosure provide an application identification method and device based on a large language model (LLM) to at least solve the problem that models in related technologies cannot adapt when facing changes in application data. When application data changes due to changes in applications, the model cannot correctly identify the application, and a large amount of labeled data is needed for training.

[0008] According to one embodiment of the present disclosure, an application identification method based on a large language model (LLM) is provided, including: analyzing the feature similarity between a plurality of application data by a large language model (LLM), and classifying the application data according to the feature similarity; training a learning model based on the classified application data to identify applications of the same category in a plurality of application data to be identified.

[0009] According to another embodiment of the present disclosure, a large language model LLM-based application identification device is provided, comprising: an analysis module configured to analyze feature similarities between a plurality of application data by a large language model LLM, and classify the application data according to the feature similarities; and an identification module configured to train a learning model based on the classified application data to identify applications of the same category in the plurality of application data to be identified.

[0010] According to still another embodiment of the present disclosure, a computer-readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when executed.

[0011] According to still another embodiment of the present disclosure, an electronic device is also provided, which comprises a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0012] According to still another embodiment of the present disclosure, a computer program product is also provided, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above method embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0013] FIG. 1 is a hardware structure block diagram of a computer terminal of a large language model LLM-based application identification method according to an embodiment of the present disclosure;

[0014] FIG. 2 is a flowchart of a large language model LLM-based application identification method according to an embodiment of the present disclosure;

[0015] FIG. 3 is another flowchart of a large language model LLM-based application identification method according to an embodiment of the present disclosure;

[0016] FIG. 4 is a schematic diagram of multi-modal data according to an embodiment of the present disclosure;

[0017] FIG. 5 is a schematic diagram of application analysis based on LLM according to an embodiment of the present disclosure;

[0018] FIG. 6 is a structure block diagram of a large language model LLM-based application identification device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0020] It should be noted that the terms “first”, “second”, etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0021] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or similar computing device. Taking the case of running on a computer terminal, FIG. 1 is a hardware structure block diagram of a computer terminal of an application identification method based on a large language model LLM according to an embodiment of the present disclosure. As shown in FIG. 1, the computer terminal can include one or more (only one is shown in FIG. 1) processors 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 configured to store data, wherein the above-mentioned computer terminal can further include a transmission device 106 configured to have a communication function and an input and output device 108. Those skilled in the art can understand that the structure shown in FIG. 1 is only schematic, which does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal can further include more or fewer components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0022] The memory 104 can be configured to store computer programs, for example, software programs of application software and modules, such as the computer program corresponding to the application identification method based on a large language model LLM in the embodiments of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the computer terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0023] The transmission device 106 is configured to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF for short) module, which is configured to communicate with the Internet in a wireless manner.

[0024] In the embodiments of the present disclosure, an application identification method based on a large language model LLM running on the above-mentioned computer terminal is provided, and FIG. 2 is a flowchart of the application identification method based on a large language model LLM according to an embodiment of the present disclosure. As shown in FIG. 2, the flow includes the following steps:

[0025] In step S202, the large language model LLM is used to analyze the feature similarity between the plurality of application data, and the application data is classified according to the feature similarity.

[0026] In this embodiment, the key features of the plurality of application data can be extracted, the feature similarity between the key features is compared, and the application data with a feature similarity greater than a threshold is classified into the same category of application. The key features at least include one of the following: keywords, visual elements, key paths, and specific traffic segments in network traffic.

[0027] In this embodiment, when the application data is multi-modal data, after the key features of the plurality of application data are extracted, the key features with a dependency relationship between modalities can be aggregated.

[0028] In step S204, the learning model is trained based on the classified application data, so as to identify the same category of application in the plurality of application data to be identified.

[0029] In this embodiment, the LLM can be trained by the classified application data to obtain a target LLM, and the target LLM is used to identify the same category of application in the plurality of application data to be identified.

[0030] In this embodiment, the classified application data can also be labeled, and the learning model is trained by the labeled application data to obtain a target learning model, and the target learning model is used to identify the same category of application in the plurality of application data to be identified. It should be noted that in this disclosure, the learning model is not limited to a machine learning model or a deep learning model.

[0031] In one embodiment, before step S202, the application data is collected, and the LLM is trained by the application data. The application data is single-modal data or multi-modal data, and the application data at least includes one of the following: text data, image data, behavior patterns, network traffic, and metadata. The text data at least includes one of the following: application name and application description. The image data at least includes one of the following: application icon and interface screenshot. The behavior pattern at least includes one of the following: login behavior and function use behavior. The network traffic includes traffic transmitted during application use. The metadata at least includes one of the following: application version information, developer information, release date, installation package size, and application signature.

[0032] Through the above steps S202-S204, the feature similarity between the plurality of application data is analyzed by the large language model LLM, and the application data is classified according to the feature similarity, and the learning model is trained based on the classified application data to identify the same category of application in the plurality of application data to be identified. The problem that the model in the prior art cannot adapt when facing application data changes, when the application data changes after the application is changed, the model cannot correctly identify the application, and a large amount of labeled data is required to train the model, and the effect of accurately and adaptively identifying the application in the network, reducing the demand for model updating, and not requiring a large amount of labeled data to train the model is achieved.

[0033] FIG. 3 is another flowchart of an application identification method based on a large language model LLM according to an embodiment of the present disclosure, as shown in FIG. 3, the flow includes the following steps:

[0034] Step S301, data collection. FIG. 4 is a schematic diagram of multi-modal data according to an embodiment of the present disclosure, as shown in FIG. 4, the data includes text data (such as application name, application description), image data (such as application icon, interface screenshot), behavior pattern (login behavior, function use behavior), network traffic (traffic transmitted during application use), and metadata related to APP (such as application version information, developer information, release date, installation package size, application signature). In order to realize the automation and efficiency of data collection, related data can be obtained by using methods such as data crawler technology, API interface calling, network sniffing, etc., and a corresponding data collection system can be established.

[0035] Step S302, extraction and analysis of application data. FIG. 5 is a schematic diagram of application analysis based on LLM according to an embodiment of the present disclosure, as shown in FIG. 5, when analyzing application data using LLM, the feature similarity between different application data needs to be mined, and applications that may be of the same origin are distinguished according to the feature similarity greater than a threshold. This process involves in-depth analysis of multi-modal data, including text, image, behavior, and network traffic data.

[0036] Text data, metadata, LLM identifies possible common themes between different application data through topic mining and other techniques. For example, if multiple applications use similar keywords, such as vocabulary, sentence patterns, etc., in their descriptions, it may imply that they have similar origins or backgrounds.

[0037] Image data, LLM can discover visual element similarities between different applications through image recognition and object detection techniques, such as icon and interface element similarities. If the icons or interface elements of multiple applications have similar design styles or structures, it may indicate that they have the same creator or origin.

[0038] Behavior patterns and network traffic, by observing the login behavior, function use behavior and application data transmission behavior of users, can find the possible pattern overlap or abnormal behavior between different applications. Extracting key paths, specific traffic segments in network traffic, the similarity of these key features may suggest the possibility that they belong to the same category of applications.

[0039] Comprehensive use of LLM to analyze multi-modal data can more comprehensively understand the similarity between different applications and classify applications belonging to the same source into one category. This in-depth analysis method provides strong support for identifying and processing applications in the same category, thereby obtaining a high-quality application identification dataset.

[0040] Step S303, identify applications in the same category. The classified application data obtained in the above steps can be combined with specific application scenarios to identify applications in the same category, such as operator networks. The operator network is a complex network ecosystem that includes core networks, access networks, wireless networks, and has a large number of users and device connections. In the operator backbone network, it is necessary to monitor network traffic in real time, ensure network security, identify and intercept malicious applications in the same category (such as horse jacket applications), and also provide value-added services to help users identify and prevent malicious applications in the same category (such as horse jacket applications) and prevent malicious software intrusion and privacy leakage.

[0041] In the operator network scenario, a trained model based on network traffic data can be used to identify the collected network traffic data to identify applications in the same category (such as horse jacket applications) and provide identification. The constructed analysis model can use machine learning, deep learning, or even LLM technology to extract features and identify patterns from network traffic data, thereby achieving accurate identification of applications in the same category (such as horse jacket applications).

[0042] First, a large amount of network traffic data is needed as a training set, which can use the high-quality application identification dataset obtained in the previous step. This data will be used to train a network traffic analysis model that can use machine learning, deep learning, or LLM technology to extract features and identify patterns from traffic data. During model construction, attention should also be paid to the accuracy and robustness of the model. After training the model, it needs to be evaluated to ensure its performance meets expectations, such as using cross-validation techniques to evaluate the performance of the model and calculating performance metrics such as accuracy, recall, and F1 score. By evaluating the performance of the model, its feasibility and effectiveness in a real environment can be determined. It should be noted that in this disclosure, the learning model is not limited to a machine learning model or a deep learning model or an LLM model.

[0043] Step S304, application deployment. The trained model is integrated into the actual application system for real-time same category application recognition, and the deployed recognition system is monitored and maintained to ensure stable operation of the system. Cloud computing technology and distributed system architecture can be used to achieve efficient deployment and management of the recognition model, while log monitoring, anomaly detection and other technical means are used to monitor the system running state in real time, discover and solve possible problems in time, and ensure the stability and reliability of the system. Through this process, network traffic data can be fully utilized to ensure network security and user privacy protection.

[0044] The above method can not only be applied to the recognition of re-jacketed applications, but also has the following applications:

[0045] 1) Application store security audit: Application stores can use this method for application security audit and screening. By identifying same category applications, it can ensure that applications in the application store are safe and reliable, thereby improving user experience and the health of the application ecosystem.

[0046] 2) Network security protection: Enterprises and organizations can apply this method to their network security systems to identify and block potential same category malicious applications. This helps prevent malicious applications from entering enterprise networks and protects sensitive data and system security.

[0047] 3) Application market regulation: Governments or regulatory agencies can use this method to regulate and manage the application market. By identifying same category applications, it can reduce the negative impact of bad applications on users, maintain market order and consumer rights and interests.

[0048] 4) Malicious behavior detection: In addition to identifying same category malicious applications, it can also be used to detect other malicious behaviors in applications, such as malicious data collection and fraud. This helps to identify and address security risks caused by malicious applications as soon as possible.

[0049] This method not only has wide application in application markets, enterprise network security and other fields, but also helps to improve user experience, protect user privacy and maintain the health of the application ecosystem, enhancing network security protection capabilities and providing strong technical support for network operation and regulation. The beneficial effects of this method mainly include the following aspects:

[0050] 1) In-depth multi-modal data analysis: Using large language models (LLM) to analyze multi-modal data of applications, including metadata, behavior patterns, network traffic, etc., which enables the recognition system to capture the characteristics of same category applications from different dimensions, improving the comprehensiveness and accuracy of recognition.

[0051] 2) Adaptation to traffic changes: Due to the adaptive learning ability of LLM, this method can cope with the continuous changes in traffic characteristics of the same category of applications without frequent updates to identification rules or model parameters, providing continuous adaptability and stability to the identification system.

[0052] 3) Reduce false negatives: Through high-quality traffic training of the same category of applications, this method can more accurately identify the traffic of the same category of applications, effectively reducing the false negative rate and improving the reliability of identification.

[0053] 4) Strengthen the identification of concealed same-category applications: For same-category applications that use advanced techniques for concealment, LLM can learn the deep patterns and behavioral characteristics of the traffic, revealing their concealed same-category applications and enhancing the depth and breadth of the identification system.

[0054] 5) Improve user security perception: For operators and government and enterprise users, this method provides an efficient same-category application identification tool, enhancing user confidence and satisfaction in network security.

[0055] Example One

[0056] Identify same-category applications based on LLM analysis + TextCNN

[0057] 1) Data collection: Collect massive amounts of multi-modal data for applications, such as specified text descriptions (metadata can be represented in text form), image data, behavioral patterns, and network traffic. Text descriptions include name, function description, installation package certificate and signature, installation package size description, etc. Image data includes icons, application internal page screenshots, etc. Behavioral patterns include login methods, function usage behaviors, etc. Network traffic includes all traffic generated during application use. These data are stored in a database and sorted and organized according to application name.

[0058] 2) LLM feature extraction: Use multi-modal LLM model to analyze the collected data. The LLM model extracts key features of each application data through analysis of these multi-modal data, such as keywords in text, visual elements in images, key paths in behavioral patterns, and specific traffic segments in network traffic.

[0059] 3) LLM analysis: Use multi-modal LLM to compare the feature similarity between key features, and divide application data with feature similarity greater than a threshold into the same category of applications. Aggregate key features that have dependent relationships between modalities. Through feature analysis and aggregation, the LLM model can identify the features before and after the same category of applications change, and aggregate and compare similar feature behaviors to identify possible same-category applications. The inducted application data is used as a label to label network traffic data.

[0060] 4) Model training: Use the labeled network traffic data to train the learning model, such as the TextCNN (Convolutional Neural Network for sentence classification) for text classification task. Through cross-validation method, the trained model is evaluated and optimized to ensure that the model has high accuracy and robustness in identifying the same category of applications.

[0061] 5) Application deployment: Deploy the trained learning model to the operator's backbone network to monitor and analyze the data traffic flowing through the network in real time. It should be noted that in the present disclosure, the learning model is not limited to machine learning models or deep learning models.

[0062] Embodiment two

[0063] Identifying the same category of applications based on end-to-end LLM

[0064] 1) Data collection, obtain massive application multi-modal data, such as specified text description (metadata can be represented in the form of text), image information, behavior patterns, network traffic four kinds, text description includes name, function description, installation package certificate and signature, installation package size description, etc., image information includes icon, application internal page screenshot, etc., behavior patterns include application use login method, application function use flow, etc., network traffic includes all traffic generated when using the application, these data are stored in the database, and classified and arranged according to the application name.

[0065] 2) Multi-modal data analysis and learning of LLM model: After data collection is completed, the data is input into the LLM model for training, for example: extract the key features of multiple application data, aggregate the key features between modalities that have dependency relationships, compare the feature similarity between key features, and divide the application data with feature similarity greater than the threshold into the same category of applications. In the training process, the LLM model not only learns the feature information of each modality data, but also can identify the data features before and after the change of the same category of applications. Specifically, through the analysis and learning of multi-modal data, the LLM model can automatically induce the same category of applications, which lays the foundation for subsequent identification of the same category of applications. In the training process, large-scale parallel computing and distributed training technologies are used to improve the training efficiency and the performance of the model. In the verification process, network traffic is used to verify the identification of the application.

[0066] 3) Model deployment and real-time identification: After the training of the LLM model is completed, it is deployed into the backbone network of the operator to realize real-time identification of network traffic. During the deployment process, the LLM model is embedded into the network monitoring system to monitor and analyze the data traffic flowing through the network in real time. In the identification stage, efficient algorithms and data structures are used to quickly process and analyze network traffic data, improving the accuracy and efficiency of identification.

[0067] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing an end device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the present disclosure.

[0068] In this embodiment, an application recognition device based on a large language model LLM is also provided, which is configured to implement the above embodiments and preferred embodiments, which have been described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0069] FIG. 6 is a structural block diagram of an application recognition device based on a large language model LLM according to an embodiment of the present disclosure, as shown in FIG. 6, the device 600 includes:

[0070] The analysis module 602 is configured to analyze the feature similarity between a plurality of application data by the large language model LLM, and classify the application data according to the feature similarity;

[0071] The recognition module 604 is configured to train a learning model based on the classified application data to identify the same category of applications in a plurality of application data to be identified.

[0072] In one embodiment, the device 600 includes:

[0073] The collection module is configured to collect application data;

[0074] The training module is configured to train the LLM using the application data;

[0075] The application data is single-modal data or multi-modal data, and the application data at least includes one of the following: text data, image data, behavior mode, network traffic, and metadata. The text data at least includes one of the following: application name and application description. The image data at least includes one of the following: application icon and interface screenshot. The behavior mode at least includes one of the following: login behavior and function use behavior. The network traffic includes traffic transmitted during application use. The metadata at least includes one of the following: application version information, developer information, release date, installation package size, and application signature.

[0076] In an embodiment, the analysis module 602 includes:

[0077] An extraction sub-module configured to extract key features of the plurality of application data, wherein the key features at least include one of the following: keyword, visual element, key path, and specific traffic segment in network traffic;

[0078] A comparison sub-module configured to compare feature similarities between the key features;

[0079] A classification sub-module configured to classify application data with a feature similarity greater than a threshold value into the same category of application.

[0080] In an embodiment, the analysis module 602 includes:

[0081] An aggregation sub-module configured to aggregate key features between modalities that have a dependency relationship.

[0082] In an embodiment, the identification module 604 includes:

[0083] A first training sub-module configured to train an LLM using the classified application data to obtain a target LLM;

[0084] A first identification sub-module configured to identify, by the target LLM, applications in the same category in the plurality of application data to be identified.

[0085] In an embodiment, the identification module 604 includes:

[0086] A labeling sub-module configured to label the classified application data with a label;

[0087] A second training sub-module configured to train a learning model using the application data labeled with the label to obtain a target learning model;

[0088] A second identification sub-module configured to identify, by the target learning model, applications in the same category in the plurality of application data to be identified. It should be noted that in the present disclosure, the learning model is not limited to a machine learning model or a deep learning model.

[0089] It should be noted that the above various modules can be implemented by software or hardware, and for the latter, the implementation can be achieved by the following ways, but is not limited to: the above modules are located in the same processor; or the above various modules are located in different processors in any combination.

[0090] Embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any of the above method embodiments when running.

[0091] In an example embodiment, the above computer readable storage medium can include, but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0092] Embodiments of the present disclosure further provide an electronic device, which comprises a memory storing a computer program and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

[0093] In an example embodiment, the above electronic device can further comprise a transmission device connected to the processor and an input / output device connected to the processor.

[0094] The specific examples in the present embodiment can refer to the examples described in the above embodiments and example implementations, and the present embodiment will not be described here again.

[0095] Embodiments of the present disclosure further provide a computer program product, which comprises computer instructions, and the computer instructions are executed by a processor to implement the steps in any of the above method embodiments.

[0096] Embodiments of the present disclosure further provide a computer program product, which comprises a non-volatile computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the steps in the method of various embodiments of the present disclosure.

[0097] It is apparent that those skilled in the art should understand that the modules or steps of the present disclosure described above can be realized by general computing devices, which can be centralized on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing devices, so that they can be stored in storage devices and executed by the computing devices, and in some cases, the steps shown or described can be executed in different orders, or they can be respectively manufactured into individual integrated circuit modules, or multiple modules or steps thereof can be manufactured into a single integrated circuit module. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0098] The preferred embodiments of the present disclosure are only used to illustrate the present disclosure, and not to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. within the principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A large language model (LLM)-based application identification method, comprising: analyzing feature similarities between a plurality of application data by a large language model (LLM), and classifying the application data according to the feature similarities; training a learning model based on the classified application data to identify applications of a same category in a plurality of application data to be identified.

2. The method of claim 1, wherein, Before analyzing feature similarities between application data by a large language model (LLM), the method comprises: collecting the application data; training the LLM with the application data; wherein the application data is single-modal data or multi-modal data, and the application data comprises at least one of the following: text data, image data, behavior patterns, network traffic, metadata; the text data comprises at least one of the following: application name, application description; the image data comprises at least one of the following: application icon, interface screenshot; the behavior patterns comprise at least one of the following: login behavior, function usage behavior; the network traffic comprises traffic transmitted during application usage; the metadata comprises at least one of the following: application version information, developer information, release date, installation package size, application signature.

3. The method of claim 1, wherein, analyzing feature similarities between a plurality of application data by a large language model (LLM), and classifying the application data according to the feature similarities comprises: extracting key features of the plurality of application data, wherein the key features comprise at least one of the following: keywords, visual elements, key paths, specific traffic segments in network traffic; comparing feature similarities between the key features; classifying application data with feature similarities greater than a threshold into applications of a same category.

4. The method of claim 3, wherein, In the case where the application data is multi-modal data, after extracting key features of the plurality of application data, the method comprises: aggregating the key features with dependent relationships between modalities.

5. The method of claim 1, wherein, training a learning model based on the classified application data to identify applications of a same category in a plurality of application data to be identified comprises: training the LLM with the classified application data to obtain a target LLM; identifying applications of a same category in the plurality of application data to be identified by the target LLM.

6. The method of claim 1, wherein, training a learning model based on the classified application data to identify applications of a same category in a plurality of application data to be identified comprises: annotating the classified application data with labels; training a learning model with the application data annotated with labels to obtain a target learning model; identifying applications of a same category in the plurality of application data to be identified by the target learning model.

7. A large language model (LLM)-based application identification apparatus, comprising: an analysis module configured to analyze feature similarities between a plurality of application data by a large language model (LLM), and classify the application data according to the feature similarities; an identification module configured to train a learning model based on the classified application data to identify applications of a same category in a plurality of application data to be identified.

8. A computer-readable storage medium having stored therein a computer program, wherein, The computer program is executed by a processor to implement the steps of the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer program product comprising a computer program, the computer program implementing the steps of the method according to any one of claims 1 to 6 when executed by a processor.

Images