Application behavior detection method and device, electronic equipment and medium

Abstract

The invention provides an application behavior detection method and device, electronic equipment and a medium, and relates to the field of computers, in particular to the field of large language models and application behavior detection. The method can comprise the steps of obtaining at least one application interface image associated with a to-be-detected application; obtaining a behavior description statement used for describing the to-be-detected behavior; and obtaining a detection result about whether the to-be-detected behavior exists in the to-be-detected application or not through a large language model based on the behavior description statement and the application interface image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to large language models and application behavior detection, and specifically to an application behavior detection method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the continuous development of mobile Internet, various applications have emerged, and various application behaviors exist in the applications. Therefore, it is hoped to obtain a solution that can detect the behaviors in the application.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides an application behavior detection method, device, electronic device, computer-readable storage medium, and computer program product.

[0005] According to one aspect of the present disclosure, there is provided an application behavior detection method, comprising: obtaining at least one application interface image associated with an application to be detected; obtaining a behavior description sentence for describing the behavior to be detected; and obtaining a detection result on whether the behavior to be detected exists in the application to be detected through a large language model based on the behavior description sentence and the application interface image.

[0006] According to another aspect of the present disclosure, there is provided an application behavior detection device, comprising: an interface image obtaining unit, for obtaining at least one application interface image associated with an application to be detected; a description sentence obtaining unit, for obtaining a behavior description sentence for describing the behavior to be detected; and a detection result obtaining unit, for obtaining, based on the behavior description sentence and the application interface image, a detection result on whether the behavior to be detected exists in the application to be detected through a large language model.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the application behavior detection method according to one or more embodiments of the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the application behavior detection method according to one or more embodiments of the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the application behavior detection method according to one or more embodiments of the present disclosure when executed by a processor.

[0010] According to one or more embodiments of the present disclosure, behaviors in applications may be effectively detected.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0014] Figure 2 A flow chart of an application behavior detection method according to an embodiment of the present disclosure is shown;

[0015] Figure 3 A schematic diagram showing a data flow of an application behavior detection method according to an embodiment of the present disclosure;

[0016] Figure 4 A structural block diagram of an application behavior detection device according to an embodiment of the present disclosure is shown;

[0017] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0019] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0020] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0021] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0023] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable the application behavior detection method according to the present disclosure to be executed.

[0024] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0025] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0026] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to perform application behavior detection, etc. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0027] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0028] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0029] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0030] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0031] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0032] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0033] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0034] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system. Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0035] Reference below Figure 2 An application behavior detection method 200 according to an exemplary embodiment of the present disclosure is described.

[0036] In step S201, at least one application interface image associated with the application to be detected is obtained.

[0037] In step S202, a behavior description statement for describing the behavior to be detected is obtained.

[0038] In step S203, based on the behavior description statement and the application interface image, a detection result on whether the behavior to be detected exists in the application to be detected is obtained through a large language model.

[0039] According to the method described in the embodiment of the present disclosure, the behavior in the application can be effectively detected.

[0040] In related fields, when detecting behaviors in applications, crawlers and other technologies may be used. However, such methods cannot accurately and comprehensively detect suspicious behaviors in applications, cannot fully simulate the scenarios in which users use apps, and the detection accuracy is not high.

[0041] According to various embodiments of the present disclosure, it is possible to utilize the large model's ability to understand images and detect suspicious behaviors or problems in applications through behavioral description statements, including but not limited to harmful content, secondary jumps to advertisements, downloads, etc.

[0042] The behavior description statement can be used to describe the characteristics or detection rules of the behavior to be detected. The behavior to be detected can be defined by the behavior description statement, so that the large model can understand and judge the application interface image based on the definition of the behavior description statement, thereby detecting whether there is a behavior to be detected. Exemplarily, the behavior to be detected can include at least one of the following: harmful content, download behavior, and jump behavior. The behavior to be detected may also include other behaviors that need to be detected, and the present disclosure is not limited to this.

[0043] Exemplarily, a rule base may be constructed. For example, the rule base may include "if xx condition is met, then it is xx harmful content". The rules corresponding to the behavior to be detected and the application interface image may be input into the large model to allow the large model to make a judgment.

[0044] According to some embodiments, obtaining at least one application interface image associated with the application to be detected may include taking a screenshot of at least one application interface of the application to be detected.

[0045] In such an example, page screenshots can be used to detect application behavior, which can fully simulate the user's perspective to detect application behavior.

[0046] Exemplarily, taking a screenshot of at least one application interface of the application to be detected may be implemented through an accessibility service.

[0047] According to some embodiments, obtaining at least one application interface image associated with the application to be detected may include: causing the application to be detected to present a first application interface; and obtaining an application interface image associated with the first application interface.

[0048] By detecting the interface images under a specific application interface, it is possible to effectively simulate various behaviors that users may exhibit when using the application.

[0049] According to some embodiments, the first application interface may include at least one of the following: an application startup page and an application landing page.

[0050] Including application startup page and landing page, covering the entire process of application startup. You can randomly call the application landing page, or visit all application landing pages.

[0051] As a specific example, when there is illegal downloading, you can access the landing page through the accessibility service, such as random or full access, and take a screenshot of the landing page. After that, you can access the download behavior rule library, combine each or several rules with the screenshot to form a prompt word, call the model service to let the model determine whether there is any download involved, and get the download behavior detection result. Figure 3The specific examples in the description are given here, and the architecture of the rule base and model service is given as an example, but it can be understood that the present disclosure is not limited to this.

[0052] According to some embodiments, the method may also include: after obtaining the application interface image associated with the first application interface, performing at least one operation in the first application interface; and after the at least one operation is performed, obtaining the application interface image associated with the current application interface of the application to be detected.

[0053] By taking screenshots after operating the interface, it is possible to detect the further behavior of the application after receiving the user operation, thereby performing a comprehensive detection of the application behavior.

[0054] For example, after the at least one operation is performed for a predetermined time, an application interface image associated with the current application interface of the application to be detected can be obtained. For example, the predetermined time can be 3 seconds, 5 seconds, etc. By operating the interface and continuing to take screenshots after waiting for a period of time, it is possible to effectively simulate user behavior and detect further behavior of the application after receiving the user operation, thereby performing a more comprehensive detection of the application behavior.

[0055] According to some embodiments, the at least one operation may be implemented through an accessibility service.

[0056] In such an embodiment, by calling the accessibility service, the function of the accessibility service to operate the page can be effectively utilized to achieve the effect of simulating user behavior, thereby effectively detecting the application.

[0057] According to some embodiments, the at least one operation may include clicking on a page. By clicking on the page, it is possible to detect whether there is any suspicious operation in the application that is triggered by the clicking behavior.

[0058] According to some embodiments, clicking on the page may include clicking on a button or text at a random position.

[0059] In this way, you can randomly click buttons or texts to simulate user behavior as much as possible and comprehensively detect problems in the application.

[0060] According to some embodiments, the at least one operation may include invoking a device shake function.

[0061] By calling the shake function of the device, the application behavior triggered by the shake function can be effectively detected.

[0062] According to some embodiments, the method may further include: obtaining a search term related to the behavior to be detected. In such an embodiment, the at least one operation may include executing a search function in the first application interface based on the search term.

[0063] You can search pages based on corresponding search terms to detect whether there is suspicious behavior in the pages. For example, you can search using search terms related to harmful content to discover hidden harmful behavior.

[0064] According to some embodiments, the at least one operation may include calling a pull-down function. By calling the pull-down function, user behavior is fully simulated, and hidden suspicious behavior can be detected.

[0065] Combine the following Figure 3 Describe the service architecture and data flow diagram according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the service architecture may include access service, simulator service, accessibility service, regular detection service, search for vocabulary list to be detected service, regular vocabulary list to be detected service, promotion behavior rule library service, harmful content rule library, download behavior rule library, secondary jump detection service, download detection service, model service, BOS service, low-quality library service, etc.

[0066] The access service can be used to call various other services through the accessibility service request.

[0067] Taking the Android platform as an example, the simulator service may be a service capable of running an Android app.

[0068] Accessibility services can be used for apps that are pre-developed and installed on the simulator service. They can actually execute search commands and get search results, and can actually execute pull-down commands and get recommended results.

[0069] The regular detection service may be a service that performs regular detection on pages, and it may detect, for example, pages with harmful content and pages with words to be detected.

[0070] The search word list service can be a service for storing search words to be detected, and MySQL can be used for storage.

[0071] The regular to-be-detected word list service can be a service for storing the to-be-detected words on a page, and can use MySQL for storage.

[0072] The promotion behavior rule library service can use MySQL for storage, and the table format can be, for example, "feature id, feature description". The feature description can be, for example, the existence of the behavior of promoting goods; the existence of promotion, sponsorship and other content, and so on.

[0073] The harmful content rule base can be stored using MySQL, and the table format can be, for example, "feature id, feature description". The feature description can include but is not limited to: "content involving xxxx", "description containing xxxx", "images containing xxxx", etc.

[0074] The download behavior rule library may be stored in MySQL. The table format may be, for example, "feature id, feature description". The feature description may include, but is not limited to: the existence of a download button, a download link, keywords such as "please click to download", and the like.

[0075] The secondary jump detection service may be a service for detecting whether a page has a secondary jump behavior.

[0076] The illegal download detection service may be a service for detecting whether a web page will induce downloading of an app.

[0077] The model service may be any large language model service that currently exists or is later launched and can be understood by those skilled in the art.

[0078] The bos service can be used to store screenshots of harmful pages that are hit.

[0079] The low-quality library service can be used to store the hazard information detected by the model. It can be a MySQL database in table format (appid, page link, screenshot address, hazard type, hazard details). The hazard details store the hit word list or rules. The types may include: word list to be detected, harmful content, promotion behavior, download behavior, etc.

[0080] As an example, the method may include running at least one of a conventional detection service process, a secondary jump detection service process, an illegal download detection service process, and a search inspection / recommendation inspection process.

[0081] Exemplarily, the regular detection service process may include the accessibility service first taking a screenshot of the first screen in the app, and then entering the landing page, such as random or full access, obtaining the text in the page in the landing page, and then calling the regular word list service for detection with the text to obtain the regular word list detection result, and then taking a screenshot of the page, accessing the harmful content rule library service, combining each or several rules with the picture to form a prompt word, calling the model service to let the model determine whether the screenshot has harmful content based on the rules, and obtaining the harmful content detection result. If low quality is detected, the screenshot will be passed to the boss service, and then the detection result will be passed to the low-quality library service for storage.

[0082] Exemplarily, the secondary jump detection service process may include entering the landing page through the accessibility service, such as random or full access, taking a screenshot of the landing page, and then the accessibility service randomly clicks the button or text on the page, waits for, for example, 5 seconds, takes a screenshot, and then accesses the promotion behavior rule library to combine each or several rules with the screenshot to splice into prompt words, calls the model service to let the model determine whether the screenshot has promotional behavior based on the rules, obtains the promotion behavior detection result, and then calls the regular detection service for regular verification, and then calls the illegal download detection service for regular verification. Similarly, for low-quality detection, the screenshot will be passed to the boss service, and then the detection result will be passed to the low-quality library service. For example, you can use the accessibility service to call the shake function of the phone, wait for 5 seconds, take a screenshot, and then access the promotion behavior rule library to combine each or several rules with the screenshot to form prompt words, call the model service to let the model determine whether the screenshot contains promotional behavior based on the rules, and get the promotion behavior detection result. Then call the regular detection service for regular verification, and then call the illegal download detection service for regular verification. Similarly, if low quality is detected, the screenshot will be passed to the boss service, and then the detection result will be passed to the low-quality library service for error reporting.

[0083] Exemplarily, the illegal download detection service process may include entering the landing page through the accessibility service (random or full access), taking a screenshot of the landing page, and then accessing the download behavior rule library to combine each or several rules with the screenshot to form prompt words, calling the model service to let the model determine whether there is any download involved, and obtaining the download behavior detection result, and then randomly clicking on the picture or button, accessing the simulator service download list to see if there is any downloaded data, and also obtaining the download behavior detection result. Similarly, for low-quality situations detected, the screenshot will be passed to the boss service, and then the detection results will be passed to the low-quality library service.

[0084] Exemplarily, the search inspection and recommendation inspection processes may include accessing the search word list service to be detected through the accessibility service to randomly obtain a word to be detected, then calling the search function of the app to obtain the home screen and some landing pages, and then calling the regular detection service, the secondary jump detection service, and the illegal download detection service. Exemplarily, the drop-down function of the app can also be directly called through the accessibility service to let the app return to the recommended home screen and some landing pages, and then call the regular detection service, the secondary jump detection service, and the illegal download detection service for detection.

[0085] According to one or more embodiments of the present disclosure, it is possible to implement an active app inspection architecture based on a big model, in-app download detection based on a big model, in-app secondary jump detection based on a big model, and detection of whether there is harmful content in the app based on a big model.

[0086] Reference now Figure 4The application behavior detection device 400 according to an embodiment of the present disclosure is described. The application behavior detection device 400 may include an interface image acquisition unit 401, a description sentence acquisition unit 402, and a detection result acquisition unit 403.

[0087] The interface image obtaining unit 401 may be used to obtain at least one application interface image associated with the application to be detected. The description sentence obtaining unit 402 may be used to obtain a behavior description sentence for describing the behavior to be detected. The detection result obtaining unit 403 may be used to obtain a detection result on whether the behavior to be detected exists in the application to be detected based on the behavior description sentence and the application interface image through a large language model.

[0088] According to the device described in the embodiment of the present disclosure, it is possible to effectively detect behaviors in applications.

[0089] According to some embodiments, the interface image obtaining unit includes a unit for taking a screenshot of at least one application interface of the application to be detected.

[0090] According to some embodiments, the interface image obtaining unit includes a unit for the following operations: causing the application to be detected to present a first application interface; and obtaining an application interface image associated with the first application interface.

[0091] According to some embodiments, the first application interface includes at least one of the following: an application startup page and an application landing page.

[0092] According to some embodiments, the device may also include a unit for the following operations: after obtaining an application interface image associated with the first application interface, performing at least one operation in the first application interface; and after the at least one operation is performed, obtaining an application interface image associated with the current application interface of the application to be detected.

[0093] According to some embodiments, the at least one operation is implemented by an accessibility service.

[0094] According to some embodiments, the at least one operation includes clicking on a page.

[0095] According to some embodiments, clicking on the page includes clicking on a button or text at a random position.

[0096] According to some embodiments, the at least one operation includes invoking a device shake function.

[0097] According to some embodiments, the device may further include a unit for obtaining a search term related to the behavior to be detected, and wherein the at least one operation includes executing a search function in the first application interface based on the search term.

[0098] According to some embodiments, the at least one operation comprises invoking a pull-down function.

[0099] According to some embodiments, the behavior to be detected includes at least one of the following: harmful content, downloading behavior, and jumping behavior.

[0100] In the technical solution disclosed herein, the collection, acquisition, storage, use, processing, transmission, provision and public application of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0101] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0102] refer to Figure 5 , a block diagram of an electronic device 500 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0103] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0104] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device that can input information to the electronic device 500. The input unit 506 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 507 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include but is not limited to a disk, an optical disk. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0105] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as method 200 and its variants, etc. For example, in some embodiments, method 200 and its variants, etc. may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method 200 and its variants, etc. described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the method 200 and its variations, etc., in any other appropriate manner (eg, by means of firmware).

[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0108] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0111] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0113] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. An application behavior detection method, comprising: Obtaining at least one application interface image associated with the application to be detected; Obtaining a behavior description sentence for describing the behavior to be detected; Based on the behavior description statement and the application interface image, a detection result on whether the behavior to be detected exists in the application to be detected is obtained through a large language model.

2. The method according to claim 1, wherein: The obtaining of at least one application interface image associated with the application to be detected includes taking a screenshot of at least one application interface of the application to be detected.

3. The method according to claim 1 or 2, wherein: The obtaining of at least one application interface image associated with the application to be detected comprises: causing the application to be detected to present a first application interface; and Obtain an application interface image associated with the first application interface.

4. The method according to claim 3, wherein: The first application interface includes at least one of the following: an application startup page and an application landing page.

5. The method according to claim 3 or 4, further comprising: After acquiring the application interface image associated with the first application interface, Execute at least one operation in the first application interface; as well as After the at least one operation is executed, an application interface image associated with the current application interface of the application to be detected is obtained.

6. The method according to claim 5, wherein: The at least one operation is implemented through an accessibility service.

7. The method according to claim 5 or 6, wherein: The at least one operation includes clicking on a page.

8. The method according to claim 7, wherein: Clicking on a page includes clicking on buttons or text at random locations.

9. The method according to any one of claims 5 to 8, wherein: The at least one operation includes invoking a shake function of the device.

10. The method according to any one of claims 5 to 9, further comprising: Obtaining search terms related to the behavior to be detected, and Among them, the at least one operation includes executing a search function in the first application interface based on the search term.

11. The method according to any one of claims 5 to 10, wherein: The at least one operation includes invoking a pull-down function.

12. The method according to any one of claims 1 to 11, wherein: The behavior to be detected includes at least one of the following: harmful content, downloading behavior, and jumping behavior.

13. An application behavior detection device, comprising: An interface image obtaining unit, used to obtain at least one application interface image associated with the application to be detected; A description statement obtaining unit, used to obtain a behavior description statement for describing the behavior to be detected; The detection result obtaining unit is used to obtain a detection result on whether the behavior to be detected exists in the application to be detected based on the behavior description statement and the application interface image through a large language model.

14. The device according to claim 13, wherein: The interface image obtaining unit includes a unit for taking a screenshot of at least one application interface of the application to be detected.

15. The device according to claim 13 or 14, wherein: The interface image obtaining unit includes units for the following operations: causing the application to be detected to present a first application interface; and Obtain an application interface image associated with the first application interface.

16. The device according to claim 15, wherein: The first application interface includes at least one of the following: an application startup page and an application landing page.

17. The apparatus according to claim 15 or 16, further comprising means for: After acquiring the application interface image associated with the first application interface, Execute at least one operation in the first application interface; and After the at least one operation is executed, an application interface image associated with the current application interface of the application to be detected is obtained.

18. The device according to claim 17, wherein: The at least one operation is implemented through an accessibility service.

19. The device according to claim 17 or 18, wherein: The at least one operation includes clicking on a page.

20. The device according to claim 19, wherein Clicking on a page includes clicking on buttons or text at random locations.

21. The device according to any one of claims 17 to 20, wherein: The at least one operation includes invoking a shake function of the device.

22. The apparatus according to any one of claims 17 to 21, further comprising a unit for obtaining a search term related to the behavior to be detected, and wherein: The at least one operation includes executing a search function in the first application interface based on the search term.

23. The device according to any one of claims 17 to 22, wherein: The at least one operation includes invoking a pull-down function.

24. The device according to any one of claims 13 to 23, wherein: The behavior to be detected includes at least one of the following: harmful content, downloading behavior, and jumping behavior.

25. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.

27. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images