Application privacy security detection method based on large language model driving

Abstract

The invention provides an application privacy security detection method based on large language model driving, relates to the technical field of network security, and is used for driving a mobile application to run automatically based on a large language model and detecting whether privacy parameters meet security requirements or not in the running process. Automatic detection can be carried out without consuming manpower, and the detection efficiency is improved. The method comprises the steps of obtaining task description information; identifying current interface structure information of the to-be-detected mobile application and acquiring a historical operation record; processing the task description information, the current interface structure information and the historical operation record for multiple times to generate a target vector representation, and inputting the target vector representation into a large language model to obtain an operation instruction prompt; prompting and controlling the to-be-detected mobile application to execute the target operation according to the operation instruction; and in the process of executing the target operation by the to-be-detected mobile application, detecting privacy parameters of the to-be-detected mobile application, and determining whether the privacy parameters meet security requirements or not.

Description

Technical Field

[0001] The present disclosure relates to the field of network security technology, and in particular to an application privacy security detection method driven by a large language model. Background Art

[0002] With the popularization of smart terminals, applications have also entered a rapid development stage. For example, there are more and more mobile applications available to users on smart mobile terminals. Since applications usually involve the collection and processing of user privacy information during operation, if such information is obtained by illegal application vendors or the collection process does not meet security requirements, it may cause serious losses to users. Therefore, it is usually necessary to detect applications.

[0003] In the related art, testers manually interact with the application to be tested to control the application operation, causing the application to expose privacy and security issues during operation.

[0004] However, this approach relies on manual interaction of testers when testing applications, requiring testers to perform a large number of identical and repetitive operations, resulting in low efficiency in application testing. Summary of the invention

[0005] The present disclosure provides an application privacy security detection method driven by a large language model, which is used to drive the application to run independently based on the large language model, and detect whether the privacy parameters meet the security requirements during the running process; automatic detection can be performed without consuming manpower, thereby improving the detection efficiency.

[0006] In order to achieve the above technical objectives, the present disclosure adopts the following technical solutions: In a first aspect, the present disclosure provides an application privacy and security detection method driven by a large language model, the method comprising: obtaining task description information, the task description information being used to describe at least one target task that the application to be detected is expected to perform, each target task involving at least one operation; identifying the current interface structure information of the application to be detected, and obtaining historical operation records; the current interface structure information is used to describe the preset page elements and the operations supported by them in the current user interface of the application to be detected in a preset structured data form; using a natural language processing model, processing the task description information, the current interface structure information and the historical operation records respectively, to obtain a first natural language text corresponding to the task description information and a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation record; inputting the first natural language text, the second natural language text and the third natural language text into a prompt generator to obtain a target vector representation output by the prompt generator, the target vector representation comprising a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text and a third semantic feature corresponding to the third natural language text; inputting the target vector representation into a large language model to obtain an operation instruction prompt output by the large language model, the operation instruction prompt being used to indicate the target operation; according to the operation instruction prompt, controlling the application to be detected to perform the target operation; in the process of the application to be detected performing the target operation, detecting the privacy parameters of the application to be detected to determine whether the privacy parameters meet the security requirements.

[0007] The method provided by the present disclosure converts the task description information, the current interface structure information of the application to be detected, and the historical operation records into a natural language description text (such as a first natural language text, a second natural language text, and a third natural language text) that is conducive to the understanding of people and large language models; further, the above texts are logically combined by a prompt generator to generate a target vector representation with a more coherent contextual relationship, which is more conducive to the understanding and processing of the large language model to output a more accurate operation instruction prompt, thereby driving the application to be detected to perform the target operation; in this way, it is ensured that the application to be detected can be accurately driven to run, and the accuracy of application detection is improved. In addition, since the parameters generated in the process (such as the first natural text language, the target vector representation, etc.) are described in natural language, when the operation of the application is abnormal, it is also beneficial for relevant personnel to quickly find the cause of the abnormality by querying the parameters generated in the process, thereby improving the detection efficiency. In addition, without consuming manpower, the application to be detected can be driven to run by itself, and the privacy parameters are detected during the operation process to see whether they meet the security requirements. In summary, the method provided by the present disclosure reduces the waste of human resources on the one hand, improves the efficiency of application detection on the other hand, and improves the accuracy of application detection on the other hand.

[0008] In some embodiments, after controlling the application to be detected to perform the target operation according to the operation instruction prompt, the above method also includes: storing the target operation in the historical operation record; when the preset termination condition is not met, returning to the step of identifying the current interface structure information of the application to be detected and obtaining the historical operation record; when the preset termination condition is met, terminating the detection of the application to be detected; wherein the preset termination condition includes at least one of the following: receiving an operation termination instruction, the operation termination instruction is used to indicate the termination of executing the operation in the application to be detected; receiving a detection completion indication, the detection completion indication is used to indicate that the detection of each privacy parameter in the application to be detected has been completed; the running time of the application to be detected reaches the preset maximum detection time; it is detected that at least one target task is fully executed; it is detected that at least one target task is executed M times, and M is a positive integer.

[0009] The method provided by the present disclosure can control the application to be detected to continue running when the operation termination condition is not met, so as to avoid insufficient detection time due to too short running time; in addition, when the operation termination condition is met, the method is no longer executed, so as to flexibly terminate the detection of the application to be detected according to different situations. In addition, the present disclosure also provides a variety of operation termination conditions; it should be understood that in the case of termination according to the operation termination instruction, it is conducive to flexibly determine when to terminate according to the user instruction; in the case of termination according to the detection completion indication, it can terminate automatically when the detection is completed to avoid the waste of computing resources; in the case of termination according to the application running time, it can avoid resource occupation caused by the application running for too long; in the case of termination detection according to at least one target task being fully executed, it can ensure that the detection is stopped only after each target task in at least one target task is executed; in the case of termination according to the number of executions of at least one target task, the maximum number of executions of each target application (for example, M) can be set to avoid falling into an infinite loop, causing the application to be detected to be run infinitely.

[0010] In some embodiments, the above-mentioned identification of the current interface structure information of the application to be detected includes: obtaining the current user interface of the application to be detected from the running platform of the application to be detected; calling the user interface processor to parse the user interface to obtain the current interface structure information of the application to be detected; the above-mentioned control of the application to be detected to perform the target operation according to the operation instruction prompt includes: converting the operation instruction prompt into a user operation instruction that can act on the user interface of the application to be detected; sending the user operation instruction to the running platform of the application to be detected, so that the running platform performs the target operation on the application to be detected according to the user operation instruction.

[0011] The method provided by the present disclosure, on the one hand, can realize the conversion from the user interface to the interface structure information by calling the user interface processor, which is conducive to the transition of information and facilitates data processing between different modules; on the other hand, the operation instruction prompt can be converted into specific user operation instructions, so that the target operation can be performed according to the user operation instructions on the running platform of the application to be detected; especially when the running platform is independent of the execution subject of the present method, it can be realized to control the operation of the application to be detected on other running platforms, which is conducive to the simultaneous detection of multiple applications to be detected.

[0012] In some embodiments, the task description information includes at least one of the following: a task serial number corresponding to each target task, which is used to identify the execution order of the target tasks; a window name corresponding to each target task, which is used to identify the name of the window view in which the target task is executed; an execution action corresponding to each target task, which is used to indicate the behavioral action required when performing the operation; and a task parameter corresponding to each target task, which is used to indicate the relevant parameters required when executing the target task.

[0013] In the method provided by the present invention, the task number can constrain the execution order of the tasks, the window name can constrain the window view of the task execution, the execution action can constrain the implementation method of the task, and the task parameters provide the relevant parameters required to execute the task; based on this, the running direction of the application to be tested is standardized through the task description information to ensure the normal operation of the application to be tested.

[0014] In some embodiments, at least one target task includes a first target task and a second target task; when at least one target task is executed according to the task sequence number corresponding to each target task, the name of the window view in which the second target task is executed is the same as the name of a window view displayed by the application to be detected after the first target task is executed; wherein the task sequence number of the second target task is the task sequence number of the first target task plus one; or, the first target task is the last target task among at least one target task, and the second target task is the first target task among at least one target task.

[0015] The method provided by the present disclosure makes the name of a window view displayed by the application to be detected after the last target task is executed the same as the name of the window view when the first target task is executed; or the name of a window view displayed by the application to be detected after a certain target task is executed is the same as the name of the window view when the next target task is executed. Thus, the execution of the at least one target task can form a cyclic closed loop, which is conducive to ensuring the continuous operation of the application to be detected.

[0016] In some embodiments, when at least one target task is executed according to the task sequence number corresponding to each target task, at least one target task can be executed cyclically; at least one target task and the task sequence number corresponding to each target task are determined based on the following method: obtaining multiple task samples of the application to be detected, each task sample corresponds to a task, the name of the window view in which the task is executed, and the name of at least one window view displayed by the application to be detected after the task is executed; inputting multiple task samples into the target task screening model to obtain at least one target task output by the target task screening model and the task sequence number corresponding to each target task.

[0017] The method provided by the present disclosure can comprehensively screen out at least one target task that can be executed cyclically and the task sequence number corresponding to each target task based on the situation after the execution of each target task (for example, the name of at least one window view displayed by the application to be detected after the execution of the target task) and the situation during the execution of each target task; thereby, it is conducive to achieving the continuous operation of the application to be detected.

[0018] In some embodiments, the above method also includes: obtaining privacy parameters of the application to be detected during the process of the application to be detected executing the target operation; wherein the privacy parameters include at least one of the following: log data generated by the application to be detected during the operation, plaintext data communicated during the operation of the application to be detected, and user permissions called by the application to be detected during the operation.

[0019] The method provided by the present disclosure detects privacy parameters including privacy parameters during the operation of the application. It should be understood that the method provided by the present disclosure can drive the application to be detected to run independently based on a large language model, and is therefore suitable for detecting privacy parameters involved in the operation of the application, which can prevent privacy data from being obtained by illegal application vendors and ensure the safe operation of the application to be detected.

[0020] In a second aspect, the present disclosure provides an application privacy and security detection device driven by a large language model, comprising: an acquisition module, an interaction module and a processing module; wherein the acquisition module is used to acquire task description information, the task description information is used to describe at least one target task that the application to be detected is expected to perform, and each target task involves at least one operation; the interaction module is used to identify the current interface structure information of the application to be detected, the current interface structure information is used to describe the preset page elements and the operations supported by the current user interface of the application to be detected in a preset structured data form; the acquisition module is also used to acquire historical operation records; the processing module is used to use a natural language processing model to process the task description information, the current interface structure information and the historical operation records respectively, and obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information, and a The processing module is further used to input the first natural language text, the second natural language text and the third natural language text into the prompt generator to obtain the target vector representation output by the prompt generator, and the target vector representation includes the first semantic feature corresponding to the first natural language text, the second semantic feature corresponding to the second natural language text and the third semantic feature corresponding to the third natural language text; the processing module is further used to input the target vector representation into the large language model to obtain the operation instruction prompt output by the large language model, and the operation instruction prompt is used to indicate the target operation; the interaction module is further used to control the application to be detected to perform the target operation according to the operation instruction prompt; the processing module is further used to detect the privacy parameters of the application to be detected during the process of the application to be detected performing the target operation to determine whether the privacy parameters meet the security requirements.

[0021] In a third aspect, the present disclosure provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer implements any one of the large language model-driven application privacy and security detection methods provided in the embodiments of the present disclosure.

[0022] In a fourth aspect, the present disclosure provides an electronic device, comprising: a processor and a memory for storing processor executable instructions; the processor is configured to execute instructions so that the electronic device implements any one of the large language model-driven application privacy and security detection methods provided in the embodiments of the present disclosure.

[0023] In a fifth aspect, the present disclosure provides a computer program product, which includes computer program instructions. When the computer instructions are executed on a computer, the computer implements any one of the large language model-driven application privacy and security detection methods provided in the embodiments of the present disclosure.

[0024] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium, wherein the computer-readable storage medium may be packaged together with the processor of the controller, or may be packaged separately from the processor of the controller, which is not limited in the present disclosure.

[0025] For the specific description of the second to fifth aspects and their various implementations in the present disclosure, reference can be made to the detailed description of the first aspect and its various implementations; and for the beneficial effects of the second to fifth aspects and their various implementations, reference can be made to the analysis of the beneficial effects in the first aspect and its various implementations, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of the structure of an application privacy and security detection system driven by a large language model provided in an embodiment of the present disclosure; Figure 2 A flowchart of an application privacy security detection method driven by a large language model provided in an embodiment of the present disclosure; Figure 3 A flowchart of another application privacy security detection method based on a large language model driven by an embodiment of the present disclosure; Figure 4 A flowchart of another application privacy security detection method driven by a large language model provided in an embodiment of the present disclosure; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0028] It should be noted that, in the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, unless otherwise specified, "multiple" means two or more.

[0029] With the popularization and promotion of smart terminals, applications have also entered a stage of rapid development. For example, users are using more and more mobile applications on smart mobile terminals. In the embodiment of the present application, taking the mobile application on the smart mobile terminal as an example, the application privacy security detection method driven by the large language model is described. However, it should be noted that the method provided in the embodiment of the present application is not limited to the detection of the privacy security of mobile applications, but can also be applications other than mobile applications, such as various applications installed on other electronic devices such as computers and smart screens.

[0030] In the field of network security technology, especially in the field of mobile application security, since mobile applications usually involve the collection and processing of user privacy information during operation, if such information is obtained by illegal application vendors or the collection process does not meet security requirements, it may cause serious losses to users. Therefore, it is usually necessary to test mobile applications. For example, in an enterprise production environment, in order to meet the privacy and security requirements of relevant departments, a mobile application needs to be tested before it is released online.

[0031] In the related art, testers manually interact with the mobile application to be tested to control the operation of the mobile application, causing the mobile application to expose privacy and security issues during operation. However, this method of testing mobile applications relies on manual interaction by testers, requiring testers to perform a large number of identical and repetitive operations, resulting in low efficiency of mobile application testing, untimely feedback, and a long rectification cycle.

[0032] In this regard, the disclosed embodiment provides an application privacy and security detection method based on a large language model drive, the method comprising: first obtaining task description information, the task description information is used to describe at least one target task that the mobile application to be detected is expected to perform, and each target task involves at least one operation; then identifying the current interface structure information of the mobile application to be detected, and obtaining historical operation records; wherein the current interface structure information is used to describe the preset page elements and the operations supported by them in the current user interface of the mobile application to be detected in a preset structured data form. In other words, the interface structure information has a preset data format, and its essence is a structured data, which describes the preset page elements in the current user interface and the operations supported by them, such as which user click operations, sliding operations, etc. can be supported by interface elements such as controls, icons, and information input boxes in the user interface. Then, the natural language processing model is used to process the task description information, the current interface structure information and the historical operation records respectively, so as to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation records; the first natural language text, the second natural language text and the third natural language text are input into a prompt generator to obtain a target vector representation output by the prompt generator, wherein the target vector representation includes a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text and a third semantic feature corresponding to the third natural language text; the target vector representation is input into a large language model to obtain an operation instruction prompt output by the large language model, wherein the operation instruction prompt is used to indicate the target operation; according to the operation instruction prompt, the mobile application to be detected is controlled to perform the target operation; in the process of the mobile application to be detected performing the target operation, the privacy parameters of the mobile application to be detected are detected to determine whether the privacy parameters meet the security requirements.

[0033] In view of this, a natural language processing model is used to convert the task description information, the current interface structure information of the mobile application to be detected, and the historical operation records into texts described in natural language that can be understood by people (such as the first natural language text, the second natural language text, and the third natural language text); further, the above texts are logically combined by a prompt generator to generate a target vector representation with a more coherent contextual relationship, which is more conducive to the understanding and processing of the large language model, so as to output a more accurate operation instruction prompt to drive the mobile application to be detected to perform the target operation; in this way, it is ensured that the mobile application to be detected can be driven and run accurately, and the accuracy of mobile application detection is improved. In addition, since the parameters generated in the process (such as the first natural text language, the target vector representation, etc.) are described in natural language, in the case of abnormal operation of the mobile application, it is also beneficial for relevant personnel to quickly find the cause of the abnormality by querying the parameters generated in the process, thereby improving the detection efficiency. In addition, without consuming manpower, the mobile application to be detected can be driven to run by itself, and the privacy parameters can be detected during the operation process to see whether they meet the security requirements. In summary, the method provided by the present disclosure reduces the waste of human resources on the one hand, improves the efficiency of mobile application detection on the other hand, and improves the accuracy of mobile application detection on the other hand.

[0034] The application privacy and security detection method based on the large language model driven provided by the present disclosure can be applied to the scenario of performing privacy and security detection during the operation of mobile applications. It should be understood that it can also be extended to any scenario that requires detection during the operation of mobile applications.

[0035] Exemplarily, in addition to the above-mentioned privacy parameters, other operating parameters may also be tested. For example, during the operation of the mobile application, it is also possible to test whether the resource parameters and / or traffic parameters of the mobile application to be tested meet the requirements. Among them, the resource parameters include at least one of the following: the storage resources actually called by the mobile application to be tested during operation, the bandwidth resources actually called by the mobile application to be tested during operation, and the computing power resources actually called by the mobile application to be tested during operation. This disclosure does not make specific limitations on this.

[0036] In order to more clearly illustrate the solution provided by the present disclosure, the present disclosure also provides a structural diagram of an application privacy security detection system driven by a large language model. Figure 1The application privacy and security detection system driven by a large language model includes at least one of the following: a natural language processor (NLP) 110, a user interface (UI) processor 120, a prompt generator 130, a large language model (LLM) 140, an operating platform 150, and a detection engine 160.

[0037] A natural language processing model is provided on the natural language processor 110. It should be understood that the natural language processing model can convert information described in other language description forms (such as structured data forms) into information described in natural language; and / or, convert information described in natural language into information described in other language description forms.

[0038] In some embodiments, the natural language processor 110 can process the task description information, the current interface structure information of the mobile application to be detected, and the historical operation records respectively to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information, and a third natural language text corresponding to the historical operation records. Thus, the task description information, the current interface structure information, and the historical operation records are natural language description text information that is easy for humans and large language models to understand.

[0039] For the related concepts of task description information, current interface structure information and historical operation records, please refer to the method embodiment below, which will not be described here. Task description information is pre-stored or obtained based on model training. The current interface structure information of the mobile application to be detected is obtained by identifying the user interface of the mobile application to be detected; for example, the current interface structure information is sent to the natural language processor 110 after being identified by the user interface processor 120. The most recent operation record in the historical operation record (for example Figure 1 The target operation shown in the figure is determined by the natural language processor 110 based on the operation instruction prompt output by the large language model 140. The relevant content of the operation instruction prompt can be referred to the description below.

[0040] In some embodiments, the natural language processor 110 can also convert the operation instruction prompt into a user operation instruction that can act on the user interface of the mobile application to be detected, and send it to the user interface processor 120 for execution.

[0041] The user interface processor 120 is used to process tasks related to the user interface of the mobile application to be detected.

[0042] In some embodiments, the user interface processor 120 is used to identify the current interface structure information of the mobile application to be detected. Exemplarily, the user interface processor 120 obtains the current user interface of the mobile application to be detected from the operating platform 150 of the mobile application to be detected, and parses the user interface to obtain the current interface structure information of the mobile application to be detected. Taking the operating platform 150 as a mobile device as an example, the user interface processor can parse the graphical user interface (GUI) of the mobile application running on the mobile device, and convert the preset page element information in the graphical user interface into structured description information.

[0043] In some embodiments, the user interface processor 120 may control the mobile application to be detected to perform a target operation according to the operation instruction prompt output by the large language model 140 .

[0044] In one example, the user interface processor 120 may convert the operation instruction prompt into a user operation instruction that can act on the user interface of the mobile application to be detected, so as to control the mobile application to be detected to perform the target operation.

[0045] In another example, when the user operation instruction is sent by the natural language processor 110, the user interface processor 120 may also control the mobile application to be detected to perform the target operation according to the user operation instruction sent by the natural language processor 110. For example, the user interface processor may also send the user operation instruction sent by the natural language processor 110 to the operating platform 150, so that the operating platform performs the target operation on the mobile application to be detected according to the user operation instruction.

[0046] The prompt generator 130 can process the first natural language text corresponding to the task description information output by the natural language processor 110, the second natural language text corresponding to the current interface structure information, and the third natural language text corresponding to the historical operation record, and convert the target vector representation, which includes the first semantic feature corresponding to the first natural language text, the second semantic feature corresponding to the second natural language text, and the third semantic feature corresponding to the third natural language text. The target vector representation can be parsed by the large language model 140. It can be understood that the prompt generator 130 extracts the semantic features of the first natural language text corresponding to the task description information, the second natural language text corresponding to the current interface structure information, and the third natural language text corresponding to the historical operation record, and logically combines them to generate prompt information (i.e., the target vector representation) with a more coherent context and containing semantic features, and the prompt information can be understood by the large language model 140. In some examples, the prompt information can also be used to query the large language model 140 for the next operation content.

[0047] The large language model 140 can process the target vector representation generated by the prompt generator 130 to obtain an operation instruction prompt, which is used to indicate the target operation. The target operation can be understood as the next operation that needs to be performed on the mobile application to be detected. Thus, the large language model 140 returns the correct action element according to the prompt generated by the prompt generator 130 (that is, the target vector representation).

[0048] Furthermore, the large language model 140 can also output operation instruction prompts to the natural language processor 110 or the user interface processor 120, so that the operation instruction prompts can be converted into user operation instructions that can act on the user interface of the mobile application to be detected through the natural language processor 110 or the user interface processor 120.

[0049] Furthermore, the target operation may be stored in a historical operation record as an input to the natural language processor 110 .

[0050] The running platform 150 is any platform capable of running the mobile application to be detected, including but not limited to terminals, user equipment, mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, as well as cellular phones, personal digital assistants (PDA), augmented reality (AR) and virtual reality (VR) devices, etc.

[0051] In some embodiments, the application privacy and security detection system driven by a large language model includes one or more operating platforms 150 , each of which runs at least one mobile application to be detected.

[0052] In some embodiments, various components of the application privacy and security detection system driven by a large language model are integrated into the operating platform 150, so that the operation and detection of the mobile application to be detected can be performed on the same platform.

[0053] In other embodiments, the operation platform 150 is independent of other components of the mobile application detection system, and the operation and detection of the mobile application to be detected can be performed on different platforms. Further, multiple mobile applications to be detected can be run on one or more platforms, and the multiple mobile applications to be detected can be uniformly detected on another platform.

[0054] For example, in the application privacy and security detection system driven by a large language model, other components except the running platform 150 are integrated on a computer, and the application privacy and security detection method driven by a large language model provided by the present disclosure is executed by the computer; the running platform 150 includes multiple smart phones; the computer implements the method provided by the present disclosure by detecting the mobile application to be detected in each of the multiple smart phones.

[0055] The detection engine 160 is used to detect the privacy parameters of the mobile application to be detected to determine whether they meet the security requirements. In some embodiments, the detection engine 160 is also provided with detection rules to determine whether each privacy parameter meets the security requirements based on the detection rules.

[0056] It should be pointed out that Figure 1 The structure shown in does not constitute a limitation on the application privacy security detection system driven by a large language model, except Figure 1 In addition to the components shown, the application privacy and security detection system may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, etc.

[0057] The execution subject of the application privacy and security detection method driven by a large language model provided in the present invention can be any device with processing functions, such as mobile phones, tablet computers, desktops, laptops, handheld computers, notebook computers, super mobile personal computers, netbooks, as well as cellular phones, personal digital assistants, augmented reality\virtual reality devices, etc.; for example, central processing units (CPU), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) and digital signal processors (DSP), etc.

[0058] In a specific example, the execution subject can be the above-mentioned detection engine 160, a terminal integrated with the detection engine 160, a collection of the above-mentioned detection engine 160 and other devices, or the above-mentioned mobile application detection system, etc., and the present disclosure does not make specific limitations on this.

[0059] The implementation of the embodiment of the present disclosure is described in detail below in conjunction with the accompanying drawings.

[0060] like Figure 2As shown, the embodiment of the present disclosure provides an application privacy security detection method driven by a large language model. The execution subject can refer to the above description. The method includes the following steps S101 to S107: S101. Obtain task description information, where the task description information is used to describe at least one target task that is expected to be executed by the mobile application to be detected, and each target task involves at least one operation.

[0061] Here, the mobile application to be detected is a mobile application that needs to be tested for privacy and security, and may also be referred to as a target mobile application, target software, target application, target program, program to be detected, mobile application to be detected, etc.

[0062] Here, the target task is a task that the mobile application to be detected is expected to complete during the process of mobile application detection. It should be understood that when an operation (such as a target operation) is performed on the mobile application to be detected to complete at least one target task, the mobile application to be detected can be automatically operated. Thus, the consumption of human resources in the process of mobile application detection is reduced and the efficiency of mobile application detection is improved.

[0063] In some embodiments, the at least one target task can be executed in a loop. Exemplarily, the at least one target task includes a first target task and a second target task; when at least one target task is executed according to the task sequence number corresponding to each target task, the name of the window view in which the second target task is executed is the same as the name of a window view displayed by the mobile application to be detected after the first target task is executed. The task sequence number of the second target task is the task sequence number of the first target task plus one; or, the first target task is the last target task among the at least one target task, and the second target task is the first target task among the at least one target task.

[0064] Based on this, the name of a window view displayed by the mobile application to be detected after executing the last target task is the same as the name of the window view when executing the first target task; or the name of a window view displayed by the mobile application to be detected after executing a certain target task is the same as the name of the window view when executing the next target task. Thus, the execution of the at least one target task can form a cyclic closed loop, which is conducive to ensuring the continuous operation of the mobile application to be detected.

[0065] Further, in the case that the at least one target task can be executed cyclically, it can be set whether to allow or support the at least one target task to be executed cyclically.

[0066] Furthermore, in the case where the at least one target task can be executed cyclically, a maximum number of cyclic executions of the at least one target task can be set to prevent the execution of the target task from falling into an infinite loop.

[0067] In other embodiments, the at least one target task mentioned above cannot be executed in a loop. For example, after executing the last target task in at least one target task, the name of at least one window view displayed by the mobile application to be detected is different from the name of the window view when the first target task in the at least one target task is executed. In other words, the execution of the at least one target task cannot form a closed loop. Based on this, at least one target task that can only be executed once can be set according to demand, such as tasks such as logging in to an account, etc.; the detection can be ended after executing each target task in at least one target task. Furthermore, in the case that the at least one target task mentioned above cannot be executed in a loop, it can be assumed that at least one target task does not support or is not allowed to be executed in a loop.

[0068] Here, the task description information is used to describe at least one target task that the mobile application to be detected is expected to execute.

[0069] In some embodiments, the task description information includes at least one of the following: a task serial number corresponding to each target task, which is used to identify the execution order of the target tasks; a window name corresponding to each target task, which is used to identify the name of the window view in which the target task is executed; an execution action corresponding to each target task, which is used to indicate the behavioral action required to execute the target task; and a task parameter corresponding to each target task, which is used to indicate the relevant parameters required to execute the target task.

[0070] The task number may also be referred to as the execution order, execution number, task number, etc. The behavior action may also be referred to as the execution action. The task parameters include the input parameters required to execute the target task, etc.

[0071] For example, taking the target task as "login page" as an example, the task description information included in the target task includes: The task number corresponding to the target task is: 1.

[0072] The window name corresponding to the target task: LoginActivity window.

[0073] Execution action corresponding to the target task: After entering the user account and password, click the Login button to log in.

[0074] The input parameters corresponding to the target task are: the account is test and the password is test.

[0075] It is not difficult to understand that the target task involves at least one operation: for example, an operation of editing a user account, an operation of editing a password, and an operation of clicking a login button.

[0076] It can be seen that the task number can constrain the execution order of the tasks, the window name can constrain the window view of the task execution, the execution action can constrain the implementation method of the task, and the task parameters provide the relevant parameters required to execute the task; based on this, the running direction of the mobile application to be tested is standardized through the task description information to ensure the normal operation of the mobile application to be tested.

[0077] In some embodiments, the form of the task description information includes but is not limited to at least one of the following: text form, table form, array form, index form, etc.

[0078] In some embodiments, the task description information is pre-stored information, for example, the task description information may be pre-stored information based on the testing experience of the tester.

[0079] In other embodiments, the task description information is information generated based on model training.

[0080] Exemplarily, when at least one target task is executed according to the task sequence number corresponding to each target task, and the at least one target task can be executed cyclically; the at least one target task and the task sequence number corresponding to each target task are determined based on the following method: obtaining multiple task samples of the mobile application to be detected, each task sample corresponds to a task, the name of the window view in which the task is executed, and the name of at least one window view displayed by the mobile application to be detected after the task is executed; inputting the multiple task samples into the target task screening model to obtain at least one target task output by the target task screening model and the task sequence number corresponding to each target task.

[0081] Based on this, it is possible to comprehensively screen out at least one target task that can be executed cyclically and the task sequence number corresponding to each target task according to the situation after the execution of each target task (for example, the name of at least one window view displayed by the mobile application to be detected after the execution of the target task) and the situation during the execution of each target task (for example, the name of the window view after the execution of the target task); thereby facilitating the continuous operation of the mobile application to be detected.

[0082] S102, identifying the current interface structure information of the mobile application to be detected and obtaining historical operation records, wherein the current interface structure information is used to describe preset page elements and supported operations in the current user interface of the mobile application to be detected in a preset structured data form.

[0083] The mobile application provides a user interface, which includes page elements, such as controls, windows, tabs, information input boxes, etc. Page elements can support user operations, such as controls support user click operations, tabs support user selection operations, windows support user open, close and zoom operations, and information input boxes support user input operations. Page elements have page element identifiers. Optionally, page elements can also have page element names. The page element identifier is unique. For example, when there are multiple controls named "buttons" in the same interface, these "buttons" have different page element identifiers but the same page element name. Similarly, operations can also have identifiers and names. When the objects of the operations are different, the identifiers corresponding to the operations with the same name are different. For example, there are multiple controls in the same interface, and click operations are performed on these controls respectively. Although the operations named "click operation" are performed on these controls, the identifiers of these "click operations" are different.

[0084] In some embodiments, the preset page elements may be all or part of the page elements in the user interface. For example, if the user considers that some of the page elements in the user interface are of high importance, these relatively important page elements may be set as preset page elements, and when determining the current interface structure information of the mobile application, only these relatively important page elements and the operations they can support are converted into the interface structure information.

[0085] In a possible implementation, a page element includes one component or multiple components. A component may be a fully or partially visible component, or a fully or partially invisible component, i.e., a hidden component. The user may set the components included in the page element according to the importance of the component. For example, a window includes a text view, an edit box, a button, a check box, an image view, etc. As a complete page element, the window is specifically composed of multiple components such as a text view component, an edit box component, a button component, a check box component, and an image view component. In this implementation, the component has a component identifier (ID) (which may also be referred to as a component number, a component ID number, or a component serial number, etc.). Optionally, the component may also have a component name. The component identifier is unique. For example, when there are multiple components named "edit boxes" in the same interface or page element, these "edit boxes" have different component identifiers but the same component name. In addition, components are also divided into types. For example, the above-mentioned view component, edit box component, and button component belong to different component types.

[0086] For a page element that includes one component, the essence of the page element is a single component. For a page element that includes multiple components, the essence of the page element is a composite of multiple components. A page element can support user operations, which can be specifically understood as some or all of the components in the page element supporting user operations. For example, the operations supported by the above window include the editing operations supported by the edit box component, the selection operations supported by the check box component, and the click operations supported by the button component.

[0087] In the embodiment of the present application, the interface structure information has a preset data format, which is essentially a structured data that describes the preset page elements in the current user interface and the operations they support.

[0088] In a possible implementation, the structured data describes the correspondence between various information related to page elements in a "tuple" data format.

[0089] For example, the correspondence between the page element identifier and the operation is described in a "binary" data format. In one example, a "binary" in the interface structure information is specifically "(A, a)". In this "binary", the first element A represents the page element identifier, which is used to indicate the page element A; the second element a is the operation identifier, which is used to indicate the operations supported by the page element A.

[0090] It should be understood that for page elements that include multiple components, it is more appropriate to use a "tuple" of data to describe the correspondence between multiple information related to a component, where the component-related information involved in the "tuple" can be defined by the user. For example, a "binary group" in the interface structure information is specifically "(B, b)", where the first element B represents the component identifier, which is used to indicate component B; the second element b is the operation identifier, which is used to indicate the operation b supported by component B. For other components in the page element, a "binary group" similar to "(B, b)" is constructed for each component. These "binary groups" are the structured data that describe this page element. The structured data corresponding to the preset page elements in an interface are combined to form the interface structure information of this interface.

[0091] Furthermore, the correspondence between multiple information related to a component can also be described through data formats such as "triplets" and "quadruplets". In one example, the correspondence between four types of information related to a component is described through the "quadruplet" data format, and a "quadruplet" in the interface structure information is specifically "(ID, name, type, operations)", where ID represents the component identifier, name represents the component name, type represents the component type, and operations represents the operations supported by the component.

[0092] In a specific embodiment, still taking the target task as the above-mentioned "login page" as an example, an active window and two button controls are provided in the user interface of the mobile application, the preset page element is the active window, and the preset data format is a "triplet" data format that describes the current interface structure information of the mobile application, wherein the first element in the "triplet" represents the identifier of a component in the page element of the current user interface of the mobile application, the second element represents the type of the component, and the third element represents the identifier of the operation that the component can support. Assuming that the active window includes a text view (TextView) component with component identification 1, an edit box (EditText) component with component identification 2, a button (Button) component with component identification 3, a check box (CheckBox) component with component identification 4, and an image view (ImageView) component with component identification 5, the current interface structure information of the mobile application is: (1, TextView, a), (2, EditText, b), (3, Button, c), (4, CheckBox, d), and (5, ImageView, e). Taking (2, EditText, b) as an example, it represents a component whose component type is an edit box component, whose component identification is 2, and which supports operation b. The same applies to other "triplets", which will not be repeated here.

[0093] In particular, from the perspective of the overall process, the "multiple group" can also be set to include the system type identifier, task identifier, interface identifier, page element identifier, etc. of the task running. For example, a "multiple group" in the interface structure information is specifically "(android, Task1, Interface1, A, ImageView1, a)", which means that the task with the task identifier Task1 running in the Android system, in a page element with the page element identifier A in the interface with the interface identifier Interface1, the component with the component identifier ImageView1 can support operation a.

[0094] In a specific embodiment, the above-mentioned identification of the current interface structure information of the mobile application to be detected includes: calling a user interface processor to parse the current user interface of the mobile application to be detected to obtain the current interface structure information. For example, calling a user interface processor to parse the current user interface of the mobile application to be detected, extracting the name of the current active window, and determining the page elements contained in the active window, thereby obtaining the current interface structure information.

[0095] When the mobile application to be detected is running on the running platform, the above-mentioned identification of the current interface structure information of the mobile application to be detected includes: obtaining the current user interface of the mobile application to be detected from the running platform of the mobile application to be detected; calling the user interface processor to parse the user interface to obtain the current interface structure information of the mobile application to be detected. Based on this, the conversion from the user interface to the interface structure information can be realized by calling the user interface processor, which is conducive to realizing the transition of information and facilitating data processing between different modules.

[0096] S103. Using a natural language processing model, the task description information, the current interface structure information and the historical operation records are processed respectively to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation records.

[0097] In some embodiments, a natural language processing model is used to extract at least one task objective and task parameter in the task description information to determine a first natural language text corresponding to the task description information.

[0098] Still taking the target task as the above-mentioned "login page" as an example, based on the task description information shown in step S101, the first natural language text of the task description information is obtained as follows: I am using a smartphone and need to interact with the LoginActivity window. After entering my user account and password, I click the Login button to log in.

[0099] In some embodiments, the current interface structure information is processed using a natural language processing model to obtain a second natural language text corresponding to the current interface structure information.

[0100] Still taking the target task as the above-mentioned "login page" as an example, if the name of the activity window identified in step S102 is "LoginActivity", the current interface structure information in the activity window is processed to obtain the second natural language text corresponding to the current interface structure information. The second natural language text obtained after processing is as follows: The name of the current active window on the phone is "LoginActivity", and the page elements included are: An input box, component number 1, component name "Please enter user name", supported operation Sa1 "click", supported operation Sa2 "edit input"; An input box, component number 2, component name "Please enter password", supported operation Sb1 "click", supported operation Sb2 "edit input"; A button, component number 3, component name "Login", supporting operation Sc1 "Click".

[0101] In some embodiments, the historical operation records are processed using a natural language processing model to obtain a third natural language text corresponding to the historical operation records.

[0102] Still taking the target task as the above-mentioned "login page" as an example, when the historical operation record includes: the component number is 1, the component name is "please enter the user name", and the component historical operation is to complete "edit input", the third natural language text corresponding to the historical operation record can be: The operations I have performed in sequence are as follows: Complete the editing input in the input box with component number 1 and component name "Please enter user name".

[0103] It should be understood that step S103 can realize the conversion of the above-mentioned structured description task description information, current interface structure information and historical operation records from structured description to natural language description. On the one hand, since the text information of natural language description is more conducive to the understanding of the large language model, the accuracy of mobile application detection can be improved; on the other hand, the text information of natural language description is also conducive to human understanding, so that the detection personnel can determine the detection status of the mobile application detection process according to the above-mentioned natural language text, and it is conducive to quickly locating the steps of the detection error.

[0104] S104. Input the first natural language text, the second natural language text, and the third natural language text into a prompt generator to obtain a target vector representation output by the prompt generator, wherein the target vector representation includes: a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text, and a third semantic feature corresponding to the third natural language text.

[0105] In a specific example, taking the natural language texts involved in the above step S103 as an example, the target vector identifier generated by the prompt generator includes the following information: I am using a smartphone and need to interact with the LoginActivity window. After entering my user account and password, I click the Login button to log in. The name of the current active window on the phone is "LoginActivity", and the page elements included are: An input box, component number 1, component name "Please enter user name", supported operation Sa1 "click", supported operation Sa2 "edit input"; An input box, component number 2, component name "Please enter password", supported operation Sb1 "click", supported operation Sb2 "edit input"; A button, component number 3, component name "Login", supported operation Sc1 "Click"; The operations I have performed in sequence are as follows: Complete the editing input in the input box with component number 1 and component name "Please enter user name"; What should I do next? Please directly return the number corresponding to the operation supported by the page element. If there is no further operation instruction, please return 0.

[0106] In summary, it can be seen that the prompt generator is able to preserve the semantic features of each natural language text, arrange the logical order of each semantic feature, and generate the corresponding target vector representation, which can be understood as a query prompt text that can be parsed by a large language model.

[0107] S105: Input the target vector representation into the large language model to obtain an operation instruction prompt output by the large language model, where the operation instruction prompt is used to indicate the target operation.

[0108] The target operation is the next operation expected to be performed in the mobile application to be detected. It should be noted that the target operation is not necessarily an operation. For example, when the target operation does not exist, that is, when the target operation is a null operation, it means that the current target task does not need to continue to perform the operation.

[0109] In a specific example, taking the target vector representation involved in step S104 as an example, if the operation instruction prompt output by the large language model is: Sb2, then the target operation corresponding to the operation instruction prompt is: edit and input in the input box with component number 2 and component name "Please enter password".

[0110] In a specific example, taking the target vector representation involved in step S104 as an example, if the operation instruction prompt output by the large language model is: 0, the current target task is completed and the operation involved in the target task does not need to be executed again.

[0111] In some embodiments, when the target operation is a null operation, the process returns to step S101 to obtain relevant information of the next target task from the task description information; when the target operation is not a null operation, the process continues to execute step S106.

[0112] S106: Control the mobile application to be detected to execute the target operation according to the operation instruction prompt.

[0113] In some embodiments, step S106 includes: converting the operation instruction prompt into a user operation instruction that can act on the user interface of the mobile application to be detected; sending the user operation instruction to the running platform of the mobile application to be detected, so that the running platform performs the target operation on the mobile application to be detected according to the user operation instruction.

[0114] In a specific example, the operation instruction prompt is converted into a user operation instruction that can be received by a smart phone device, and the user operation instruction is sent to the smart phone device running the mobile application to be detected through an application programming interface (API), so that the smart phone device performs the target operation on the mobile application to be detected according to the user operation instruction.

[0115] Based on this, the operation instruction prompt can be converted into specific user operation instructions so that the target operation can be performed according to the user operation instructions on the running platform of the mobile application to be detected; especially when the running platform is independent of the execution subject of this method, it is possible to control the operation of the mobile application to be detected on other running platforms, which is conducive to the simultaneous detection of multiple mobile applications to be detected.

[0116] S107 . During the process of the mobile application to be detected executing the target operation, the privacy parameters of the mobile application to be detected are detected to determine whether the privacy parameters meet the security requirements.

[0117] The privacy parameters include at least one of the following: log data generated during the operation of the mobile application to be detected, plaintext data communicated during the operation of the mobile application to be detected, and user permissions called during the operation of the mobile application to be detected.

[0118] The application privacy security detection method based on the large language model drive provided by the present disclosure uses a natural language processing model to convert task description information, the current interface structure information of the mobile application to be detected, and the historical operation record into a natural language description text (such as a first natural language text, a second natural language text, and a third natural language text) that is conducive to the understanding of people and the large language model; further, the above texts are logically combined by a prompt generator to generate a target vector representation with a more coherent contextual relationship, which is more conducive to the understanding and processing of the large language model to output a more accurate operation instruction prompt, thereby driving the mobile application to be detected to perform the target operation; in this way, it is ensured that the mobile application to be detected can be accurately driven to run, and the accuracy of mobile application detection is improved. In addition, since the parameters generated in the process (such as the first natural text language, the target vector representation, etc.) are described in natural language, when the operation of the mobile application is abnormal, it is also beneficial for relevant personnel to quickly find the cause of the abnormality through the parameters generated in the query process, thereby improving the detection efficiency. In addition, without consuming manpower, the mobile application to be detected can be driven to run by itself, and the privacy parameters can be detected during the operation to see if they meet the security requirements. In summary, the method provided by the present disclosure reduces the waste of human resources on the one hand, improves the efficiency of mobile application detection on the other hand, and further improves the accuracy of mobile application detection on the other hand.

[0119] In some embodiments, Figure 3 As shown, after the above step S107, the following steps S108 to S110 are also included: S108: Store the target operation in the historical operation record.

[0120] The historical operation record may also be called a historical operation list, a historical operation sequence set, etc.

[0121] S109: Determine whether a preset termination condition is met.

[0122] The preset termination condition includes at least one of the following: Condition 1: A run termination instruction is received, where the run termination instruction is used to instruct to terminate the execution of an operation in the mobile application to be detected.

[0123] It should be understood that in the case of termination according to a running termination instruction, it is helpful to flexibly determine when to terminate according to a user instruction.

[0124] Condition 2: A detection completion indication is received, where the detection completion indication is used to indicate that the detection of each privacy parameter in the mobile application to be detected has been completed.

[0125] It should be understood that in the case of termination according to the detection completion indication, it can terminate automatically when the detection is completed to avoid wasting computing resources.

[0126] Condition 3: The running time of the mobile application to be detected reaches the preset maximum detection time.

[0127] It should be understood that in the case of termination according to the running length of the mobile application, resource occupation caused by the excessively long running of the mobile application can be avoided.

[0128] Condition 4: It is detected that at least one target task is fully executed.

[0129] It should be understood that, in the case where the detection is terminated based on the at least one target task being fully executed, it is possible to ensure that the detection is stopped only after all target tasks in the at least one target task are executed.

[0130] Condition 5: It is detected that at least one target task is executed M times, where M is a positive integer.

[0131] It should be understood that, in the case of termination according to at least one target task execution number, a maximum execution number (eg, M) of each target mobile application may be set to avoid falling into an infinite loop, causing the mobile application to be detected to run infinitely.

[0132] Based on this, a variety of conditions for terminating the execution of the application privacy security detection method driven by a large language model are provided. It should be understood that in actual applications, there may be more possible preset termination conditions, which are not limited in this disclosure.

[0133] If the preset termination condition is not met, return to step S102; S110: When a preset termination condition is met, terminate the detection of the mobile application to be detected.

[0134] It should be understood that if the preset termination condition is not met, executing the target operation may cause the user interface of the mobile application to be detected to change, so returning to step S102 can re-identify the user interface to improve the accuracy of mobile application detection.

[0135] Taking the preset termination condition including the above condition 4 as an example, when all target tasks in at least one task have not been fully executed, the above steps S102 to S110 are repeatedly executed according to the task sequence number until all target tasks are fully executed.

[0136] based on Figure 3 The embodiment shown can control the mobile application to be detected to continue running when the running termination condition is not met, so as to avoid insufficient detection time due to too short running time; in addition, when the running termination condition is met, the method is no longer executed, so as to flexibly terminate the detection of the mobile application to be detected according to different situations.

[0137] In some embodiments, the above method also includes: sending exception prompt information when an exception occurs in the execution of the target operation; wherein the exception prompt information includes at least one of the following: a first natural language text, a second natural language text, a third natural language text, a target vector representation, and an operation instruction prompt.

[0138] It should be understood that since text information described in natural language is more in line with human description, it is easier for people to understand, so that detection personnel can determine the detection status of the mobile application detection process based on abnormal temporary information and quickly locate the steps where the detection errors occur.

[0139] In order to more clearly illustrate the application privacy security detection method based on large language model driven provided by the present disclosure, the present disclosure also shows a specific example. Figure 4 As shown, the method comprises the following steps: S201, obtaining task description information; using a natural language processing model to process the task description information, to obtain a first natural language text corresponding to the task description information.

[0140] S202: Using a user interface processor to identify current interface structure information of the mobile application to be detected.

[0141] S203: Process the current interface structure information using a natural language processing model to obtain a second natural language text corresponding to the current interface structure information.

[0142] S204, obtaining historical operation records; using a natural language processing model to process the historical operation records to obtain a third natural language text corresponding to the historical operation records.

[0143] S205 . Input the first natural language text, the second natural language text, and the third natural language text into a prompt generator to obtain a target vector representation output by the prompt generator.

[0144] S206: Input the target vector representation into the large language model to obtain an operation instruction prompt output by the large language model, where the operation instruction prompt is used to indicate the target operation.

[0145] S207, determine whether the target operation indicated by the operation instruction is a no-operation; if not, execute the following step S208; if so, execute the following step S211.

[0146] S208: Calling a user interface processor to convert the operation instruction prompt into a user operation instruction that can act on the user interface of the mobile application to be detected.

[0147] S209: Send the user operation instruction to the running platform of the mobile application to be detected, so that the running platform performs the target operation on the mobile application to be detected according to the user operation instruction.

[0148] S210: Store the target operation in the historical operation record, and return to the above step S202.

[0149] S211. Determine whether the at least one target task is fully executed. If so, terminate the execution of the application privacy security detection method driven by a large language model; if not, execute step S212.

[0150] S212: Obtain the next target task among the at least one target task, and return to execute step S201.

[0151] Based on this, the task description information, the current interface structure information of the mobile application to be detected, and the historical operation records are respectively converted into texts of natural language descriptions that are conducive to the understanding of people and large language models; further, the above texts are logically combined by the prompt generator to generate a target vector representation with a more coherent contextual relationship, which is more conducive to the understanding and processing of the large language model to output a more accurate operation instruction prompt, thereby driving the mobile application to be detected to perform the target operation; in this way, it is ensured that the mobile application to be detected can be accurately driven to run, and the accuracy of mobile application detection is improved. In addition, since the parameters generated in the process (such as the first natural text language, the target vector representation, etc.) are described in natural language, in the case of abnormal operation of the mobile application, it is also beneficial for relevant personnel to quickly find the cause of the abnormality through the parameters generated in the query process, thereby improving the detection efficiency. In addition, without consuming manpower, the mobile application to be detected can be driven to run by itself, and the privacy parameters can be detected during the operation process to see whether they meet the security requirements. In summary, the method provided by the present disclosure reduces the waste of human resources on the one hand, improves the efficiency of mobile application detection on the other hand, and improves the accuracy of mobile application detection on the other hand.

[0152] It can be seen that the above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the method. In order to achieve the above functions, the embodiment of the present disclosure provides a hardware structure and / or software module corresponding to each function. It should be easily appreciated by those skilled in the art that, in combination with the modules and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0153] The embodiment of the present disclosure also provides an application privacy and security detection device driven by a large language model, which is used to execute the method provided by the present disclosure. The application privacy and security detection device driven by a large language model includes: an acquisition module, an interaction module and a processing module. In some embodiments, the mobile application detection device also includes a storage module.

[0154] The acquisition module is used to acquire task description information, where the task description information is used to describe at least one target task that the mobile application to be detected is expected to perform, and each target task involves at least one operation.

[0155] The interactive module is used to identify the current interface structure information of the mobile application to be detected. The current interface structure information is used to describe the preset page elements and the operations supported by them in the current user interface of the mobile application to be detected in a preset structured data form.

[0156] In some embodiments, the acquisition module is also used to acquire historical operation records.

[0157] The processing module is used to use a natural language processing model to process the task description information, the current interface structure information and the historical operation records respectively, to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation records.

[0158] In some embodiments, the processing module is also used to input the first natural language text, the second natural language text, and the third natural language text into the prompt generator to obtain a target vector representation output by the prompt generator, the target vector representation including a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text, and a third semantic feature corresponding to the third natural language text.

[0159] In some embodiments, the processing module is further used to input the target vector representation into the large language model to obtain an operation instruction prompt output by the large language model, where the operation instruction prompt is used to indicate the target operation.

[0160] In some embodiments, the interaction module is further used to control the mobile application to be detected to perform a target operation according to the operation instruction prompt.

[0161] In some embodiments, the processing module is further used to detect the privacy parameters of the mobile application to be detected during the process of the mobile application to be detected performing the target operation, and determine whether the privacy parameters meet the security requirements.

[0162] In some embodiments, the storage module is used to store the target operation in a historical operation record.

[0163] In some embodiments, the processing module is also used to return to the step of identifying the current interface structure information of the mobile application to be detected and obtaining the historical operation record when the preset termination condition is not met; when the preset termination condition is met, terminate the detection of the mobile application to be detected; wherein, please refer to the above description for the preset termination condition.

[0164] In some embodiments, the acquisition module is further used to acquire the current user interface of the application to be detected from the running platform of the mobile application to be detected.

[0165] In some embodiments, the processing module is further used to call the user interface processor to parse the user interface to obtain the current interface structure information of the mobile application to be detected.

[0166] In some embodiments, the interaction module is specifically used to convert operation instruction prompts into user operation instructions that can act on the user interface of the mobile application to be detected; and send the user operation instructions to the operating platform of the mobile application to be detected, so that the operating platform performs the target operation on the mobile application to be detected according to the user operation instructions.

[0167] In some embodiments, when at least one target task is executed according to the task sequence number corresponding to each target task, at least one target task can be executed cyclically; the processing module is also used to determine at least one target task and the task sequence number corresponding to each target task based on the following method: obtaining multiple task samples of the mobile application to be detected, each task sample corresponds to a task, the name of the window view in which the task is executed, and the name of at least one window view displayed by the mobile application to be detected after the task is executed; inputting multiple task samples into the target task screening model to obtain at least one target task output by the target task screening model and the task sequence number corresponding to each target task.

[0168] It should be noted that the above division of the modules in the device is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or in the form of software function modules.

[0169] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiment of the present disclosure also provides a structural schematic diagram of an electronic device. Figure 5 As shown, the electronic device 500 includes: a processor 502 and a bus 504. Optionally, the electronic device may further include a memory 501; and optionally, the electronic device may further include a communication interface 503.

[0170] The processor 502 may be a processor that implements or executes various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor 502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor 502 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0171] The communication interface 503 is used to connect to other devices via a communication network, which may be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0172] The memory 501 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0173] As a possible implementation, the memory 501 may exist independently of the processor 502, and the memory 501 may be connected to the processor 502 via a bus 504, and is used to store instructions or program codes. When the processor 502 calls and executes the instructions or program codes stored in the memory 501, the application privacy security detection method based on the large language model driven provided in the embodiment of the present disclosure can be implemented.

[0174] In another possible implementation, the memory 501 may also be integrated with the processor 502 .

[0175] The bus 504 may be an extended industry standard architecture (EISA) bus, etc. The bus 504 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0176] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device can be divided into different functional modules to complete all or part of the functions described above.

[0177] The embodiment of the present disclosure also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by computer instructions to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be the memory or memory of any of the above embodiments. The above computer-readable storage medium can also be the above external storage device based on the electronic device, such as the plug-in hard disk, smart memory card (smart media card, SMC), secure digital (secure digital, SD) card, flash card (flash card), etc. equipped on the above electronic device. Further, the above computer-readable storage medium can also include both the internal storage unit and / or external storage device of the above electronic device. In addition, the computer-readable storage medium can be used to store computer program products and other programs and data required by the above electronic device. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0178] The embodiments of the present disclosure also provide a computer program product. When the computer program product is run on a computer, the computer is enabled to execute any one of the large language model-driven application privacy and security detection methods provided in the above embodiments.

[0179] Although the present disclosure is described herein in conjunction with various embodiments, in the process of implementing the present disclosure for which protection is claimed, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. The term "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple components. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0180] Although the present disclosure has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present disclosure. Accordingly, this specification and the drawings are merely exemplary illustrations of the present disclosure as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present disclosure. Obviously, those skilled in the art may make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

[0181] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A large language model-driven application privacy security detection method, characterized in that: The method comprises: Acquire task description information, where the task description information is used to describe at least one target task that the application to be detected is expected to perform, and each of the target tasks involves at least one operation; Identify the current interface structure information of the application to be detected and obtain historical operation records; the current interface structure information is used to describe the preset page elements and the operations supported by them in the current user interface of the application to be detected in a preset structured data form; Using a natural language processing model, the task description information, the current interface structure information and the historical operation record are processed respectively to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation record; Inputting the first natural language text, the second natural language text, and the third natural language text into a prompt generator, obtaining a target vector representation output by the prompt generator, wherein the target vector representation includes: a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text, and a third semantic feature corresponding to the third natural language text; Inputting the target vector representation into a large language model to obtain an operation instruction prompt output by the large language model, wherein the operation instruction prompt is used to indicate a target operation; According to the operation instruction prompt, control the application to be detected to perform the target operation; During the process of the application to be detected executing the target operation, the privacy parameters of the application to be detected are detected to determine whether the privacy parameters meet the security requirements.

2. The method according to claim 1, characterized in that After controlling the application to be detected to perform the target operation according to the operation instruction prompt, the method further includes: storing the target operation in the historical operation record; If the preset termination condition is not met, returning to the step of identifying the current interface structure information of the application to be detected and obtaining the historical operation record; When the preset termination condition is met, terminating the detection of the application to be detected; The preset termination condition includes at least one of the following: receiving an operation termination instruction, wherein the operation termination instruction is used to instruct to terminate the execution of the operation in the application to be detected; Receiving a detection completion indication, where the detection completion indication is used to indicate that detection of each privacy parameter in the application to be detected has been completed; The running time of the application to be detected reaches a preset maximum detection time; detecting that the at least one target task is fully executed; It is detected that the at least one target task is executed M times, where M is a positive integer.

3. The method according to claim 1, characterized in that The identifying the current interface structure information of the application to be detected includes: Acquire the current user interface of the application to be detected from the running platform of the application to be detected; Calling a user interface processor to parse the user interface to obtain current interface structure information of the application to be detected; The step of controlling the application to be detected to perform the target operation according to the operation instruction prompt includes: Converting the operation instruction prompt into a user operation instruction that can act on the user interface of the application to be detected; The user operation instruction is sent to the running platform of the application to be detected, so that the running platform performs the target operation on the application to be detected according to the user operation instruction.

4. The method according to claim 1, characterized in that: The task description information includes at least one of the following: The task sequence number corresponding to each of the target tasks, the task sequence number is used to identify the execution order of the target tasks; The window name corresponding to each of the target tasks, the window name is used to identify the name of the window view in which the target task is executed; An execution action corresponding to each of the target tasks, wherein the execution action is used to indicate a behavioral action required when executing the operation; The task parameters corresponding to each of the target tasks are used to indicate the relevant parameters required when executing the target tasks.

5. The method according to claim 4, characterized in that The at least one target task includes a first target task and a second target task; when the at least one target task is executed according to the task sequence number corresponding to each of the target tasks, the name of the window view in which the second target task is executed is the same as the name of a window view displayed by the application to be detected after the first target task is executed; The task number of the second target task is the task number of the first target task plus one; or The first target task is the last target task among the at least one target task, and the second target task is the first target task among the at least one target task.

6. The method according to claim 4, characterized in that In the case where the at least one target task is executed according to the task sequence numbers corresponding to the target tasks, the at least one target task can be executed cyclically; the at least one target task and the task sequence numbers corresponding to the target tasks are determined based on the following method: Acquire multiple task samples of the application to be detected, each of the task samples corresponding to a task, a name of a window view in which the task is executed, and a name of at least one window view displayed by the application to be detected after the task is executed; The plurality of task samples are input into a target task screening model to obtain the at least one target task output by the target task screening model and a task sequence number corresponding to each of the target tasks.

7. The method according to claim 1, characterized in that The method further comprises: In the process of the application to be detected performing the target operation, obtaining the privacy parameters of the application to be detected; The privacy parameter includes at least one of the following: log data generated during the operation of the application to be detected, plaintext data communicated during the operation of the application to be detected, and user permissions called during the operation of the application to be detected.

8. An application privacy security detection device driven by a large language model, characterized in that: include: An acquisition module, used for acquiring task description information, wherein the task description information is used for describing at least one target task expected to be performed by the application to be detected, each of the target tasks involving at least one operation; An interaction module, used to identify the current interface structure information of the application to be detected, wherein the current interface structure information is used to describe the preset page elements and the operations supported by them in the current user interface of the application to be detected in a preset structured data form; The acquisition module is also used to acquire historical operation records; A processing module, used to process the task description information, the current interface structure information and the historical operation record respectively by using a natural language processing model to obtain a first natural language text corresponding to the task description information, a second natural language text corresponding to the current interface structure information and a third natural language text corresponding to the historical operation record; The processing module is further used to input the first natural language text, the second natural language text, and the third natural language text into a prompt generator to obtain a target vector representation output by the prompt generator, wherein the target vector representation includes a first semantic feature corresponding to the first natural language text, a second semantic feature corresponding to the second natural language text, and a third semantic feature corresponding to the third natural language text; The processing module is further used to input the target vector representation into the large language model to obtain an operation instruction prompt output by the large language model, wherein the operation instruction prompt is used to indicate the target operation; The interaction module is further used to control the application to be detected to perform the target operation according to the operation instruction prompt; The processing module is further configured to detect the privacy parameters of the application to be detected during the process of the application to be detected executing the target operation, and determine whether the privacy parameters meet the security requirements.

9. An electronic device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions so that the electronic device performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images