Card generation method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are a card generation method and apparatus, and a device and a storage medium. The method comprises: performing content extraction on a first interface, so as to obtain original interface data of the first interface; preprocessing the original interface data of the first interface, so as to obtain service data of the first interface; determining a target card template that matches the service data; filling the target card template with the service data, so as to obtain a target card of the first interface; and displaying the target card of the first interface. In this way, key service data is acquired on the basis of interface display content, and is automatically converted into a card for display, and key information in different applications is collected and displayed, without the need to rely on a third-party application to provide source data, thereby improving user experience.

Description

A method, apparatus, device and storage medium for generating cards

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to a prior Chinese patent application, application number 202411913254.2, filed on December 23, 2024, entitled "A Card Generation Method, Apparatus, Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of terminal technology, and in particular to a card generation method, apparatus, device and storage medium. Background Technology

[0004] Currently, electronic devices have a large number of applications installed. In addition to displaying application icons on the desktop, key information about the applications is presented on the desktop in the form of cards to enable users to quickly view the key information of the applications.

[0005] In related technologies, card creation relies on the access of third-party applications, which require key information to generate cards. This card generation method is limited by the third-party applications, and the card styles are limited. Summary of the Invention

[0006] This application aims to provide a card generation method, apparatus, device, and storage medium.

[0007] The technical solution of this application is implemented as follows:

[0008] Firstly, a card generation method is provided, including:

[0009] Extract content from the first interface to obtain the original interface data of the first interface;

[0010] The original interface data of the first interface is preprocessed to obtain the business data of the first interface;

[0011] Identify the target card template that matches the business data;

[0012] The business data is populated into the target card template to obtain the target card on the first screen;

[0013] Display the target card on the first screen.

[0014] Secondly, a card generating apparatus is provided, comprising:

[0015] The acquisition unit is configured to extract content from the first interface to obtain the original interface data of the first interface;

[0016] The preprocessing unit is configured to preprocess the original interface data of the first interface to obtain the business data of the first interface.

[0017] The generation unit is configured to determine the target card template that matches the business data; the business data is then filled into the target card template to obtain the target card for the first interface;

[0018] The display unit is configured to display the target card of the first interface.

[0019] Thirdly, an electronic device is provided, comprising: a display, a processor, and a memory configured to store a computer program capable of running on the processor.

[0020] When the processor is configured to run a computer program, the steps of the aforementioned method are executed.

[0021] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned method.

[0022] Fifthly, a computer program product includes a computer program that, when executed by a processor, implements the steps of the aforementioned method.

[0023] This application provides a card generation method, apparatus, device, and storage medium. The method includes: extracting content from a first interface to obtain raw interface data; preprocessing the raw interface data to obtain business data; determining a target card template matching the business data; filling the target card template with the business data to obtain a target card for the first interface; and displaying the target card for the first interface. Thus, by obtaining business data from the interface based on its displayed content and converting it into a matching card format, the method eliminates the need for source data from third-party applications, enabling the collection and display of key information from different applications and improving user experience. Attached Figure Description

[0024] Figure 1 is a schematic diagram of the first process of the card generation method in an embodiment of this application;

[0025] Figure 2 is a schematic diagram of the second process of the card generation method in an embodiment of this application;

[0026] Figure 3 is a schematic diagram of the extraction process of the image and text extraction framework in the embodiments of this application;

[0027] Figure 4 is a schematic diagram of the data acquisition process in an embodiment of this application;

[0028] Figure 5 is a schematic diagram of the data preprocessing process in an embodiment of this application;

[0029] Figure 6 is a schematic diagram of digital-modular matching during card generation in an embodiment of this application;

[0030] Figure 7 is a schematic diagram of the composition structure of the card generation device in an embodiment of this application;

[0031] Figure 8 is a schematic diagram of the composition structure of the electronic device in the embodiment of this application. Detailed Implementation

[0032] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0033] Figure 1 is a schematic diagram of the first process of the card generation method in an embodiment of this application. As shown in Figure 1, the method may specifically include:

[0034] Step 101: Extract content from the first interface to obtain the original interface data of the first interface;

[0035] The first interface can be the current display interface of the electronic device. Specifically, the first interface can be the display interface of the currently running application, which includes any application installed on the electronic device.

[0036] The original interface data specifically includes the interface layout information and interface element information of the first interface. The interface layout information includes the relationship between interface elements. The interface element information specifically includes the attributes, position, text, images, URLs, etc. of the interface elements. The attribute information includes, but is not limited to, the interface element tags, categories, titles, styles, etc.

[0037] In some embodiments, content extraction of the first interface to obtain the original interface data of the first interface includes: obtaining the interface identifier of the first interface; based on the interface identifier of the first interface, calling the interface extraction function of the software development kit to obtain the view tree of the first interface; and based on the view tree of the first interface, determining the original interface data of the first interface.

[0038] For example, data cleaning can be performed on the view tree of the first interface to obtain the original interface data of the first interface. Data cleaning can remove unnecessary view information, view structure, and other data in advance.

[0039] For example, the view tree of the first interface can be used as the original interface data.

[0040] In some embodiments, content extraction is performed on the first interface to obtain the original interface data of the first interface, including: obtaining the interface identifier of the first interface; based on the interface identifier of the first interface, calling the interface extraction function of the software development kit (SDK) to obtain the view tree of the first interface; and based on the view tree of the first interface, determining the original interface data of the first interface.

[0041] In some embodiments, content extraction of the first interface to obtain the original interface data of the first interface includes: taking a screenshot of the first interface to obtain a screenshot of the first interface; and performing image recognition on the screenshot of the first interface based on an image recognition algorithm to extract the original interface data of the first interface.

[0042] In some embodiments, content extraction of the first interface to obtain the original interface data of the first interface includes: obtaining the interface identifier of the first interface; based on the interface identifier of the first interface, calling the screenshot function of the software development kit to obtain a screenshot of the first interface; and performing image recognition on the screenshot of the first interface based on an image recognition algorithm to extract the original interface data of the first interface.

[0043] Based on the interface identifier of the first interface, determine whether the interface type of the first interface belongs to the interface types that can be extracted by the SDK. If the interface type of the first interface belongs to the interface types that can be extracted by the SDK, then call the interface extraction function of the SDK to extract the original interface data of the first interface based on the interface identifier. If the interface type of the first interface does not belong to the interface types that can be extracted by the SDK, then call the screenshot function of the SDK to obtain a screenshot of the first interface, and perform image recognition on the screenshot of the first interface based on the image recognition algorithm to extract the original interface data of the first interface.

[0044] For example, the SDK provides functionality to extract content (text, images, etc.) from native controls in third-party applications. The caller passes the interface identifier of the first screen (e.g., the activity name) to the SDK interface. The SDK then searches for the corresponding ViewTree using the passed activityName. If the passed parameter is null, it searches for the top-level ViewTree (which can be understood as the ViewTree of the currently displayed screen). Different types of Views within the ViewTree return different types of data. For example, TextView returns text; ImageView returns an image; SurfaceView returns a screenshot; TextureView returns a screenshot; and WebView returns a DOM tree, further subdividing element types and data. It should be noted that the returned screenshots or images can be further analyzed and parsed using image recognition algorithms to extract key information.

[0045] In some embodiments, the method further includes: when the first interface is displayed, obtaining the interface identifier of the first interface; and triggering the execution of a card generation method based on the interface identifier of the first interface.

[0046] Interface identifiers are used to uniquely identify a single interface (or screen, window) of an application. For example, interface identifiers include, but are not limited to, Activity Name. In the Android system, an Activity identifies a single interface in an application, and the Activity Name is used to uniquely identify this single interface in the code.

[0047] For example, a card generation method triggered based on the interface identifier of a first interface may include: determining that the first interface is allowed to generate cards based on the interface identifier of the first interface and a first list, and triggering the card generation method to be executed, wherein the first list includes the interface identifiers of one or more interfaces that are allowed to generate cards.

[0048] In other words, the interface identifiers of one or more interfaces that are allowed to present cards are obtained in advance, and a first list is constructed using these interface identifiers. That is, the first list can be used as an interface whitelist. If the interface identifier of the first interface is detected to belong to the first list, the card generation action for the first interface is automatically triggered to obtain the target card of the first interface. If the interface identifier of the first interface is detected not to belong to the first list, the card generation action for the first interface is skipped.

[0049] In some embodiments, the method further includes: in response to a user setting operation, setting an enabling state for a target interface identifier in the first list, wherein the enabling state includes enabling or disabling. When the target interface identifier is enabled, it indicates that the user allows the system to automatically trigger a card generation action for the target interface; when the target interface identifier is disabled, it indicates that the user prohibits the system from automatically triggering a card generation action for the target interface. In other words, the user can also flexibly configure the first list to achieve personalized settings for card generation actions based on interface content.

[0050] In some embodiments, the method may further include: when a first interface is displayed, obtaining a first operation on the first interface; and triggering the execution of a card generation method in response to the first operation on the first interface. The first operation may be a card generation operation performed by the user on the first interface, that is, card generation on the first interface may also be triggered by the user performing a specific operation according to their own needs.

[0051] In some embodiments, the method may further include: when a first interface is displayed, acquiring a first operation on the first interface; in response to the first operation on the first interface, determining, based on the interface identifier of the first interface and a first list, that the first interface is allowed to generate cards, and triggering the execution of a card generation method. That is, when the card generation action on the first interface is triggered by the user, to ensure user data security and the card generation effect, it is necessary to determine whether the first interface is allowed to generate cards based on the first list.

[0052] In some embodiments, based on the interface identifier of the first interface and the first list, it is determined that the first interface is allowed to generate cards, and the first interface is controlled to display the target control or display the target control as enabled; a first operation is obtained for the target control; in response to the first operation for the target control, the card generation method is triggered to be executed; based on the interface identifier of the first interface and the first list, it is determined that the first interface is prohibited from generating cards, and the first interface is controlled not to display the target control or display the target control as disabled.

[0053] Step 102: Preprocess the original interface data of the first interface to obtain the business data of the first interface;

[0054] Preprocessing transforms the raw interface data into business data with a standard data structure to improve data quality and make it more suitable for subsequent card template matching and card generation. Business data includes key business data displayed on the first interface, and includes text, images, URLs, and other data required to be presented in card format from the original interface data.

[0055] In some embodiments, the original interface data of the first interface is preprocessed to obtain the business data of the first interface, including: classifying the first interface into business types based on the interface identifier of the first interface and / or the original interface data of the first interface; determining the extraction strategy of the first interface based on the business types and mapping relationships of the first interface, wherein the mapping relationship includes extraction strategies corresponding to at least one business type; and extracting the business data of the first interface from the original interface data of the first interface based on the extraction strategy.

[0056] The extraction strategy defines the key data that must be extracted from the first interface and the extraction rules. For example, the extraction strategy includes: required data fields, optional data fields, and extraction rules.

[0057] Required data fields include required text, image URLs, and video URLs, while optional data fields include optional text, image URLs, and video URLs.

[0058] Extraction rules refer to the rules and methods used in data processing and text analysis to extract desired information from raw data. By using extraction rules, specific content can be extracted accurately and efficiently, resulting in a standardized data structure. For example, extraction rules can be one of the following: extraction rules based on regular expressions, extraction rules based on fixed formats, extraction rules based on semantic understanding, or extraction rules based on machine learning.

[0059] Based on the extraction strategy, the business data of the first interface is extracted from the original interface data of the first interface, including: extracting the required data fields and / or optional data fields from the original interface data based on the extraction rules to obtain the business data of the first interface.

[0060] For example, construct regular expressions based on required and optional data fields; use regular expressions to extract required and / or optional data fields from the original interface data to obtain the business data of the first interface.

[0061] For example, a fixed-format extraction rule is constructed based on required and optional data fields. The required and / or optional data fields in the original interface data are extracted using the specified extraction rule to obtain the business data of the first interface.

[0062] For example, semantic understanding-based extraction rules can be used to semantically understand the original interface data to obtain the business data of the first interface, which includes required data fields, optional data fields, and their related fields.

[0063] For example, semantic understanding of the original interface data is performed based on machine learning extraction rules to obtain the business data of the first interface. In some embodiments, required data fields and optional data fields can be used as targets for machine learning.

[0064] Step 103: Determine the target card template that matches the business data;

[0065] In some embodiments, a target card template matching the business data is determined by selecting a template based on the business data using a target model; or, a target card template matching the business data is determined based on the mapping relationship between business type and card template.

[0066] The target model can be a large model with template selection capabilities. In some embodiments, the target model designs suitable target templates for business data through analysis and understanding. In other embodiments, the target model determines the target card template from a variety of preset card templates based on the business data. By leveraging the generalization capability of the target model, it adapts to the card presentation requirements of different business data, thereby ensuring the effective presentation of cards for different business types.

[0067] The mapping relationship between business types and card templates defines the card template that matches each business type. The business type is determined by analyzing and understanding business data, and the target card template is selected from a variety of preset card templates based on the business type.

[0068] The service type is determined based on the identification information and / or service data on the first interface. For example, service types include: notifications, messages, orders, trips, weather, exercise, battery level, playback progress, etc. Different service types have different card presentation requirements, and therefore require different card templates.

[0069] Step 104: Fill the target card template with business data to obtain the target card for the first interface;

[0070] In some embodiments, the data structure of the business data is transformed into the target data structure defined by the target card template to obtain the target card of the first interface. For example, the target data structure includes, but is not limited to, JSON format, XML format, HTML format, etc.

[0071] For example, the target model extracts the key data required for the target card template from the business data, and accurately fills the key data into the corresponding positions in the target card template to obtain the target card containing the content of the first interface. In other words, the target model can also have a data model matching function, which accurately fills the business data into the target card template after the target card template is determined.

[0072] Step 105: Display the target card on the first screen.

[0073] In some embodiments, the method further includes: when a change in the business data of the first interface is detected, acquiring the changed data of the first interface; and updating the business data of the first interface based on the changed data of the first interface.

[0074] In some embodiments, the method further includes: generating simulated control information for a first interface in response to a second operation on a target card; and performing a control operation on the first interface based on the simulated control information for the first interface.

[0075] It's important to note that since the target cards are generated based on the interface content and do not rely on third-party applications, user actions on the target cards can be simulated by the system, allowing for quick and easy access to the first interface and eliminating the tedious process of opening applications to search for information. For example, clicking on a target card leads to the first interface, and clicking on a control on the target card controls the corresponding control on the first interface.

[0076] In some embodiments, the method further includes: storing business data in a database; reading business data from the database when the display conditions of the business data are met; filling the business data into a target card template to obtain a target card for the first interface; and displaying the target card for the first interface.

[0077] By adopting the above technical solution, business data of the interface is obtained based on the content displayed on the interface, and the business data is transformed into matching card format for presentation. This eliminates the need to rely on third-party applications to provide source data, enabling the collection and display of key information from different applications and improving user experience.

[0078] To better illustrate the purpose of this application, further examples are provided based on the above embodiments. As shown in Figure 2, the method specifically includes:

[0079] Step 201: Triggering, specifically, triggered by user screenshots, automatically triggered by a specified page (such as generating a card based on the screen content after a user buys movie tickets, tickets, plane tickets, train tickets, or subscribes to a certain service), or triggered by a specified user action (such as performing a specific action on the user on the current screen).

[0080] Step 202: Data collection. Specifically, the original interface data is obtained through the image and text extraction framework, or the original interface data is obtained by performing image recognition on the screenshot of the first interface or the image contained in the first interface through an image recognition algorithm.

[0081] The data acquisition phase is responsible for obtaining raw data from the interface, mainly through an image and text extraction framework to extract mobile screen content. This framework provides extraction of Native control content (text, images) from third-party applications, extraction of interface layout information from AndroidView and WebView, and, when encountering self-drawn components, obtaining a screenshot of the interface through screenshot processing, and calling an image recognition algorithm to recognize the screenshot and obtain the raw data. Figure 3 is a schematic diagram of the extraction process of the image and text extraction framework in this embodiment of the application. As shown in Figure 3, the extraction process of the image and text extraction framework is as follows:

[0082] (1) The caller passes the activityName of the specified interface to the ViewExtractSDK interface to extract the content of the corresponding interface. If the parameter is null, i.e., no interface is specified, the content of the top-level interface is extracted. The ViewExtractSDK interface specifically includes:

[0083] `requestViewExtractTree(parameters include callback, bundle, activityName)`: Retrieves ViewTree data; this interface is used to retrieve the view tree data of a specified Activity. An Activity is typically considered a single interface within an application, and the view tree manages the view hierarchy of the user interface. Each Activity has a corresponding ViewTree, which is a tree-like structure built from the root node using combinations of Views and ViewGroups. The view tree contains all visible and interactive elements in the application.

[0084] `getBitmapBytViewUniqueId(parameters include callback, activityName, bundle, uniqueId, parcelFileDescriptor)`: Retrieves the image content of a native control (such as a custom view or a view of a specific type). The image content of a native control refers to the image data contained in or displayed in user interface controls (such as buttons, text boxes, image views, etc.) developed using native code (such as Java, Kotlin, or C++) in Android or other mobile operating systems. These native controls are typically embedded in the application's interface to provide visual feedback and interaction to the user.

[0085] getBitmapByViewViewUniqueIds(parameters include callback, activityName, bundle, uniqueIds, parcelFileDescriptors): Retrieves the content of multiple images from a native control;

[0086] requestSpecificViewExtract(parameters include callback, bundle, activityName): Retrieves the content of a specific View; unlike requestViewExtractTree, this interface may allow finer-grained control, such as extracting only specific subviews or views with specific attributes.

[0087] (2) Calling system services through OSDK interfaces:

[0088] The Activity Task Manager Service (ATMS) is responsible for finding the corresponding viewroot by passing in the activityName. If it is null, it will find the top-level viewroot.

[0089] The extraction framework service is responsible for distinguishing the input parameters and obtaining the corresponding interface data.

[0090] (3) The system loads the target application process, obtains the ViewRootImpl object and ViewTree of the target activity, and then parses the ViewTree. Different types of Views in the ViewTree return different types of data, for example...

[0091] TextView: Returns text;

[0092] ImageView: Returns an image;

[0093] Draw the SurfaceView: Return screenshot;

[0094] Draw the view (TextureView): Returns a screenshot;

[0095] WebView: Returns the DOM tree, with detailed element types and data;

[0096] 4-5) Data is transferred from the target application process to the system process and then to the calling process.

[0097] 6) The data is received by the OnViewExtractCallback callback interface as the result of the ViewExtractSDK extraction operation and returned to the caller.

[0098] Figure 4 is a schematic diagram of the data acquisition process in an embodiment of this application. As shown in Figure 4, the data acquisition method may specifically include:

[0099] Step 401: The caller triggers the call to the SDK interface;

[0100] Step 402: Obtain current interface information;

[0101] For example, the current interface information can be a package or activity name. It is determined whether the current interface information is in the first list. The first list predefines the package or activity name of one or more interfaces that are allowed to generate cards.

[0102] Step 403: Determine if the information on the current interface is in the first list. If yes, proceed to step 404; otherwise, it indicates that card generation is prohibited on the current interface, and the data collection process ends.

[0103] Step 404: Determine whether the current interface information is in the OCR whitelist. If yes, proceed to step 407; otherwise, proceed to step 405.

[0104] Step 405: Call the SDK interface to extract the original interface data (text + images);

[0105] It should be noted that if the original interface data includes images, the image data can also be extracted using OCR.

[0106] Step 406: OCR extracts image data;

[0107] Step 407: Call the SDK interface to obtain a screenshot of the interface;

[0108] Step 408: OCR extracts screenshot data

[0109] Step 409: Determine if the data extraction was successful; if yes, proceed to step 410; if no, proceed to step 411.

[0110] Step 410: Data preprocessing; if data preprocessing fails, proceed to step 411.

[0111] Step 411: Initiate the retry mechanism and re-extract data from the current interface.

[0112] In some embodiments, if calling the SDK interface to extract the original interface data fails, the interface screenshot data can also be extracted through OCR.

[0113] Step 203: Preprocess the original interface data to obtain the business data of the first interface; extract key data from the original interface data through information extraction techniques such as extraction model and regular expression, and convert it into a standard data structure to improve data quality and make it more suitable for subsequent card template matching and card generation.

[0114] Figure 5 is a schematic diagram of the data preprocessing process in an embodiment of this application. As shown in Figure 5, the data preprocessing method specifically includes:

[0115] Step 501: Data preprocessing begins;

[0116] Step 502: Classify business operations based on the original interface data and determine the business type;

[0117] Step 503: Determine the extraction strategy based on the business type;

[0118] The extraction strategy defines the required data fields, optional data fields, and extraction rules for the current business type. The extraction rules can be based on regular expressions or on a fixed format.

[0119] Step 504: Extract business data using regular expressions or fixed rules;

[0120] Step 505: Extract business data through the extraction model

[0121] In some embodiments, the extraction rules can also be machine learning-based extraction rules, which input the raw interface data into the trained extraction model and output business data.

[0122] Step 506: Business data extraction successful. If yes, proceed to step 508; otherwise, proceed to step 507.

[0123] Step 507: Extraction failed, retry mechanism initiated;

[0124] In some embodiments, if regular expressions or fixed rules fail to extract business data, an extraction model or other extraction methods can be used for extraction; if the extraction model fails, regular expressions, fixed rules or other extraction methods can be used for extraction.

[0125] Step 508: Business data storage;

[0126] Step 509: Data needs to be updated. If yes, proceed to step 508 to update the stored business data; otherwise, end.

[0127] Step 204: Store the business data from the first interface into the database;

[0128] Step 205: Card generation;

[0129] It should be noted that the card generation stage is responsible for selecting a matching target card template for the preprocessed business data, transforming the business data into the data structure defined by the target card template, and obtaining the target card. Card generation includes two key technologies: template selection and data-model matching.

[0130] Template selection can be achieved through technologies such as Large Language Model (LLM) to select a matching target card template based on the characteristics of business data.

[0131] 1) Predefine card templates

[0132] 2) Select template through large model

[0133] 3) Data Model Matching: As shown in Figure 6, key information can be extracted from business data using LLM technology and accurately filled into the target card template. After selecting a template, since the data structure defined in the template differs from the business data, fields need to be accurately selected from the business data and filled into the corresponding "slots" in the template.

[0134] This application embodiment combines large model capability cards, automatically converting important information displayed on the electronic device interface into cards, and then presenting them naturally, ensuring the card's adaptability and presentation effect.

[0135] Step 206: Card presentation.

[0136] Step 207: User interaction.

[0137] The system can simulate the user's actions on the first screen when the user clicks on the target card, quickly realizing the operation on the first screen and eliminating the tedious process of the user opening the application to find information.

[0138] By employing the above technical solution, for individuals who face a large volume of information to process daily, artificial intelligence technology can automatically generate cards representing important information of interest to users and promptly remind them, significantly improving the user experience. Furthermore, when key information is displayed in card format, it supports one-click access by clicking the card, eliminating the cumbersome process of opening the application to search for information.

[0139] To implement the method of the embodiments of this application, based on the same inventive concept, the embodiments of this application also provide a card generation apparatus, as shown in FIG7. The card generation apparatus 70 includes:

[0140] The acquisition unit 701 is configured to extract content from the first interface to obtain the original interface data of the first interface;

[0141] The preprocessing unit 702 is configured to preprocess the original interface data of the first interface to obtain the business data of the first interface.

[0142] The generation unit 703 is configured to determine the target card template that matches the business data; and to fill the business data into the target card template to obtain the target card on the first interface.

[0143] Display unit 704 is configured to display the target card of the first interface.

[0144] In some embodiments, the card generation apparatus 70 further includes: a triggering unit configured to, when displaying a first interface, acquire an interface identifier of the first interface; and trigger the execution of a card generation method based on the interface identifier of the first interface.

[0145] In some embodiments, the triggering unit is configured to determine, based on the interface identifier of the first interface and a first list, that the first interface is allowed to generate cards, and trigger the execution of a card generation method, wherein the first list includes the interface identifiers of one or more interfaces that are allowed to generate cards.

[0146] In some embodiments, the triggering unit is configured to acquire a first operation on the first interface when the first interface is displayed; and to trigger the execution of a card generation method in response to the first operation on the first interface.

[0147] In some embodiments, the acquisition unit 701 is configured to acquire the interface identifier of the first interface; based on the interface identifier of the first interface, call the interface extraction function of the software development kit to acquire the view tree of the first interface; and based on the view tree of the first interface, determine the original interface data of the first interface.

[0148] In some embodiments, the acquisition unit 701 is configured to perform screenshot processing on the first interface to obtain a screenshot of the first interface; and to perform image recognition on the screenshot of the first interface based on an image recognition algorithm to extract the original interface data of the first interface.

[0149] In some embodiments, the acquisition unit 701 is configured to acquire the interface identifier of the first interface; based on the interface identifier of the first interface, call the screenshot function of the software development kit to acquire a screenshot of the first interface; and perform image recognition on the screenshot of the first interface based on an image recognition algorithm to extract the original interface data of the first interface.

[0150] In some embodiments, the preprocessing unit 702 is configured to classify the first interface into business types based on the interface identifier and / or the original interface data of the first interface, and determine the business type of the first interface; determine the extraction strategy of the first interface based on the business type and mapping relationship of the first interface, wherein the mapping relationship includes extraction strategies corresponding to at least one business type; and extract the business data of the first interface from the original interface data of the first interface based on the extraction strategy.

[0151] In some embodiments, the extraction strategy includes: required data fields, optional data fields, and extraction rules. The preprocessing unit 702 is configured to extract the required data fields and / or optional data fields from the original interface data based on the extraction rules, thereby obtaining the business data of the first interface.

[0152] In some embodiments, the generation unit 703 is configured to select a target card template that matches the business data based on the target model; or, based on the mapping relationship between business type and card template, determine a target card template that matches the business data.

[0153] In some embodiments, the preprocessing unit 702 is further configured to acquire the changed data of the first interface when a change in the business data of the first interface is detected; and update the business data of the first interface based on the changed data of the first interface.

[0154] In some embodiments, the card generation apparatus 70 further includes: a control unit configured to generate analog control information for a first interface in response to a second operation on a target card; and to perform a control operation on the first interface based on the analog control information for the first interface.

[0155] In practical applications, the aforementioned device can be an electronic device or a chip configured as an electronic device. In this application, the device can implement the functions of multiple units through software, hardware, or a combination of both, enabling the device to execute the card generation method provided in any of the above embodiments. Furthermore, the technical effects of each technical solution of the device can be referenced to the technical effects of the corresponding technical solutions in the card generation method, and will not be elaborated upon further in this application.

[0156] Based on the hardware implementation of each unit in the above-mentioned device, this application embodiment also provides an electronic device, as shown in FIG8. The electronic device 80 includes: a display 801, a processor 802 and a memory 803 configured to store a computer program capable of running on the processor.

[0157] When the processor 802 is configured to run a computer program, it executes the method steps described in the foregoing embodiments.

[0158] Of course, in practical applications, the various components in this electronic device 80 are coupled together via a bus system. It can be understood that the bus system is used to achieve communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus systems in the diagram.

[0159] In practical applications, the aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic device used to implement the above processor functions can also be other types, and the embodiments of this application do not specifically limit this.

[0160] The aforementioned memory can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0161] In an exemplary embodiment, this application also provides a computer-readable storage medium, such as a memory including a computer program, which can be executed by a processor of an electronic device to perform the steps of the aforementioned method.

[0162] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the methods in this application.

[0163] Optionally, the computer program product can be applied to the electronic device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0164] This application also provides a computer program.

[0165] Optionally, the computer program can be applied to the electronic device in the embodiments of this application. When the computer program is run on a computer, it causes the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0166] It should be understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0167] It should be understood that the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. The expressions “having,” “may have,” “comprising,” and “including,” or “may include” and “may contain” used herein may be used to indicate the presence of a corresponding feature (e.g., an element such as a numerical value, function, operation, or component), but do not exclude the presence of additional features.

[0168] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0169] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and devices can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0171] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0172] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0173] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Industrial applicability

[0174] This application provides a card generation method, apparatus, device, and storage medium. The method includes: extracting content from a first interface to obtain raw interface data; preprocessing the raw interface data to obtain business data; determining a target card template matching the business data; filling the target card template with the business data to obtain a target card for the first interface; and displaying the target card for the first interface. Thus, by obtaining business data from the interface based on its displayed content and converting it into a matching card format, the method eliminates the need for source data from third-party applications, enabling the collection and display of key information from different applications and improving user experience.

Claims

1. A card generation method, wherein, The method includes: Extract content from the first interface to obtain the original interface data of the first interface; The original interface data of the first interface is preprocessed to obtain the business data of the first interface; Identify the target card template that matches the business data; The business data is filled into the target card template to obtain the target card of the first interface; The target card is displayed on the first interface.

2. The method according to claim 1, wherein, The method further includes: When the first interface is displayed, obtain the interface identifier of the first interface; Based on the interface identifier of the first interface, the card generation method is triggered.

3. The method according to claim 2, wherein, The method of triggering the card generation method based on the interface identifier of the first interface includes: Based on the interface identifier of the first interface and the first list, it is determined that the first interface is allowed to generate cards, and the card generation method is triggered. The first list includes the interface identifiers of one or more interfaces that are allowed to generate cards.

4. The method according to any one of claims 1-3, wherein, The method further includes: When the first interface is displayed, obtain a first operation for the first interface; In response to a first operation on the first interface, the card generation method is triggered.

5. The method according to claim 1, wherein, The step of extracting content from the first interface to obtain the original interface data of the first interface includes: Obtain the interface identifier of the first interface; Based on the interface identifier of the first interface, the interface extraction function of the software development kit is invoked to obtain the view tree of the first interface. Based on the view tree of the first interface, determine the original interface data of the first interface.

6. The method according to claim 1 or 5, wherein, The step of extracting content from the first interface to obtain the original interface data of the first interface includes: Take a screenshot of the first interface to obtain a screenshot of the first interface; The screenshot of the first interface is processed using an image recognition algorithm to extract the original interface data.

7. The method according to claim 1, wherein, The preprocessing of the original interface data of the first interface to obtain the business data of the first interface includes: Based on the interface identifier of the first interface and / or the original interface data of the first interface, the first interface is classified into business categories to determine the business type of the first interface. Based on the business type and mapping relationship of the first interface, an extraction strategy for the first interface is determined, wherein the mapping relationship includes an extraction strategy corresponding to at least one business type. Based on the extraction strategy, the business data of the first interface is extracted from the original interface data of the first interface.

8. The method according to claim 7, wherein, The extraction strategy includes: required data fields, optional data fields, and extraction rules; The step of extracting business data of the first interface from the original interface data of the first interface based on the extraction strategy includes: Based on the extraction rules, the required data fields and / or optional data fields in the original interface data are extracted to obtain the business data of the first interface.

9. The method according to claim 1, wherein, The process of determining the target card template that matches the business data includes: The target model selects a template based on the business data to determine a target card template that matches the business data; or... Based on the mapping relationship between business type and card template, a target card template that matches the business data is determined.

10. The method according to claim 1, wherein, The method further includes: If a change is detected in the business data of the first interface, the changed data of the first interface is obtained; Update the business data on the first interface based on the changed data on the first interface.

11. The method according to claim 1, wherein, The method further includes: In response to a second operation on the target card, simulated control information for the first interface is generated; Based on the simulated control information for the first interface, execute control operations for the first interface.

12. A card generating apparatus, wherein, The device includes: The acquisition unit is configured to extract content from the first interface to obtain the original interface data of the first interface; The preprocessing unit is configured to preprocess the original interface data of the first interface to obtain the business data of the first interface. The generation unit is configured to determine a target card template that matches the business data; and to fill the business data into the target card template to obtain the target card of the first interface. The display unit is configured to display the target card of the first interface.

13. An electronic device, wherein, Electronic devices include: a display, a processor, and memory configured to store computer programs capable of running on the processor. Wherein, when the processor is configured to run the computer program, it performs the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images