Feature data acquisition method and related product thereof

Abstract

The invention discloses a feature data acquisition method and a related product thereof. The method comprises the following steps: receiving a feature configuration file issued by a server; collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and in response to a data transmission instruction sent by the server, reporting and feeding back the real-time feature data to the server, so that the server can determine a processing strategy based on the real-time feature data. The real-time performance and the accuracy of the feature data can be improved, the simplicity and the convenience of feature data acquisition are enhanced, and the calculation burden of the server is reduced.

Description

Technical Field

[0001] The present application generally relates to the field of data processing technology. More specifically, the present application relates to a feature data acquisition method and related products. Background Art

[0002] At present, widely used data stream processing technologies include Apache Kafka, Apache Flink, Spark Streaming, etc. These technologies mainly run in large-scale distributed environments, can process large amounts of data streams in real time, and support real-time analysis and decision-making. However, the traditional way of acquiring feature data has the following challenges, which brings great challenges to the improvement of subsequent data analysis effects: First, in terms of feature timeliness, the traditional way of acquiring behavior data requires the use of tracking points, and the process of uploading feature data to the cloud server by tracking points usually has a delay of minutes. In addition, the process of cloud servers parsing through real-time computing engines usually has task delays, which further leads to delays in real-time feature data. Second, in terms of feature usability, for solutions that use tracking points to acquire behavior data, any tracking point changes need to rely on client releases or business releases, with long iteration cycles and low R&D efficiency. At the same time, there is a lack of unified maintenance of user behavior, equipment, time and space, and other features, which makes feature data inconvenient to use. Third, in terms of feature computing resource consumption, in the current data link, when all user behaviors are associated, multiple data need to be aggregated, especially the aggregation of real-time data streams, which requires a large amount of computing resources of the cloud server and increases the computing burden of the cloud server.

[0003] In view of this, there is an urgent need to provide a feature data collection method so as to improve the real-time and accuracy of feature data, enhance the simplicity and convenience of feature data collection, and reduce the computing burden of the server. Summary of the invention

[0004] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a feature data collection method and related products in multiple aspects. The feature data collection method can improve the real-time and accuracy of feature data, enhance the simplicity and convenience of feature data collection, and reduce the computing burden of the server.

[0005] In the first aspect, the present application provides a feature data collection method, including: receiving a feature configuration file issued by a server; collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and responding to a data transmission instruction issued by the server, reporting the real-time feature data back to the server, so that the server can determine a processing strategy based on the real-time feature data.

[0006] In some embodiments, collecting real-time feature data based on a feature configuration file includes: parsing the feature configuration file to obtain feature configuration information; wherein the feature configuration information includes feature collection rules, feature definition information, and feature reporting rules; updating local configuration information based on the feature configuration information; intercepting preset hit events to obtain event raw data; determining whether the event raw data meets preset conditions; and if so, collecting real-time feature data from the event raw data based on the feature configuration information.

[0007] In some embodiments, parsing the feature configuration file includes: determining whether the feature configuration file has a historical download record based on a configuration file link in the feature configuration file; if not, reading the feature configuration file to obtain feature version information and initial configuration information; and based on the feature version information and local version information, removing configuration information in the initial configuration information that does not conform to the local version information.

[0008] In some embodiments, the preset hit events include tracking events and system behavior events; wherein, intercepting the preset hit events includes: intercepting the tracking events through an interceptor; and / or intercepting the system behavior events through a hook program.

[0009] In some embodiments, determining whether the original event data meets the preset conditions includes: converting the original event data into dictionary form data; determining whether the dictionary parameters in the dictionary form data are within a preset value range; and if so, determining that the preset conditions are met.

[0010] In some embodiments, before reporting the real-time feature data back to the server in response to a data transmission instruction issued by the server, the method further includes: enclosing the real-time feature data in the reporting network request.

[0011] In some embodiments, enclosing the real-time characteristic data in the network reporting request includes: determining a path whitelist for each reporting interface corresponding to each actual service in the network reporting request based on the real-time characteristic data and each service characteristic requirement corresponding to each actual service; and enclosing the real-time characteristic data in the network reporting request based on the path whitelist of each reporting interface.

[0012] In some embodiments, receiving the feature configuration file sent by the server includes: receiving the feature configuration file sent by the server through a configuration sending platform.

[0013] In a second aspect, the present application provides a device for feature data collection, comprising: a memory; and at least one processor, which is configured to: receive a feature configuration file issued by a server; collect real-time feature data based on the feature configuration file, and store the real-time feature data in a client database; and report the real-time feature data back to the server in response to a data transmission instruction issued by the server, so that the server can determine a processing strategy based on the real-time feature data.

[0014] In a third aspect, the present application provides a non-volatile machine-readable medium having program code stored thereon for collecting feature data. When the program code is executed by at least one processor, the code guides the execution operation of the at least one processor. The program code includes: receiving a feature configuration file sent by a server; collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and reporting the real-time feature data back to the server in response to a data transmission instruction issued by the server, so that the server can determine a processing strategy based on the real-time feature data.

[0015] The technical solution provided by this application may have the following beneficial effects:

[0016] The feature data collection method and related products provided by the present application realize the dynamic update of feature configuration without re-issuing the client application by receiving the feature configuration file issued by the server, thereby greatly shortening the iteration cycle, improving the development efficiency, and strengthening the simplicity and convenience of feature data collection. Then, based on the feature configuration file, real-time feature data is collected, so that the real-time collection of feature data can be realized through the client, the feature data transmission delay is reduced, the timeliness and accuracy of the feature data can be ensured, and the local computing power of the client can be used to reduce the computing burden of the server. In addition, the real-time feature data is stored in the client database, and the real-time feature data is uniformly stored and managed, ensuring the integrity, consistency and traceability of the data, reducing the number of transmissions of real-time feature data on the network, effectively reducing the risk of data leakage, and enhancing data privacy protection. Then, in response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server, so that the server can determine the processing strategy based on the real-time feature data, thereby providing efficient data support and decision-making basis for actual business, and improving the quality and efficiency of business processing.

[0017] In general, this application can improve the real-time and accuracy of feature data, enhance the simplicity and convenience of feature data collection, and reduce the computing burden of the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0019] Figure 1 An exemplary flow chart of a method for collecting feature data according to some embodiments of the present application is shown;

[0020] Figure 2 An exemplary flow chart showing a method for collecting feature data according to other embodiments of the present application is shown;

[0021] Figure 3 An exemplary flow chart showing a method for collecting feature data according to some further embodiments of the present application is shown;

[0022] Figure 4 The block diagram shows the hardware configuration of a device 400 for feature data collection that can implement the feature data collection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. For the simplicity and clarity of the description, the figure marks may be repeated in the drawings to indicate corresponding or similar elements when deemed appropriate. In addition, the present application sets forth many specific details in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other cases, well-known methods, processes, and components are not described in detail to avoid blurring the embodiments described herein. Moreover, the description should not be regarded as limiting the scope of the embodiments described herein. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0024] It should be understood that the possible terms "first" or "second" etc. in the claims, specifications and drawings disclosed in this application are used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of this application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0025] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] The traditional method of acquiring feature data has the following challenges, which poses a great challenge to improving the subsequent data analysis effect: First, in terms of feature timeliness, there is usually a delay of minutes in the process of uploading feature data to the cloud server. In addition, there is usually a task delay in the process of cloud servers parsing through real-time computing engines, which further leads to delays in real-time feature data. Second, in terms of feature usability, any changes to the tracking points need to rely on client version release or business release, with a long iteration cycle and low R&D efficiency. At the same time, there is a lack of unified maintenance of user behavior, equipment, time and space, and feature data is inconvenient to use. Third, in terms of feature computing resource consumption, when aggregating multiple data, especially the aggregation of real-time data streams, a large amount of computing resources of the cloud server is required, which increases the computing burden of the cloud server.

[0028] In view of this, there is an urgent need to provide a feature data collection method so as to improve the real-time and accuracy of feature data, enhance the simplicity and convenience of feature data collection, and reduce the computing burden of the server.

[0029] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0030] Figure 1 An exemplary flow chart 100 showing a method for collecting characteristic data in some embodiments of the present application is shown in FIG. Figure 1 , the characteristic data collection method shown in the embodiment of the present application may include:

[0031] In step S101, a feature configuration file sent by the server is received. In an embodiment of the present application, the server can send the feature configuration file to the client through a configuration sending platform, and the client receives it. The aforementioned configuration sending platform can be an ABT (A / B Testing) platform or a message platform. Among them, the A / B Testing platform is a tool for scientific experiments and decision-making, which allows different ideas, strategies or functions to be tested by comparing two or more versions (usually version A and version B). The purpose of this test is to determine which version performs better on specific indicators, such as key performance indicators (KPIs) such as click-through rate, conversion rate, and user retention. It is an important tool for data-driven decision-making and product optimization.

[0032] The ABT platform can be used to configure features to form feature configuration files. Specifically, information such as feature collection rules, feature definition information, and feature reporting rules can be integrated to form a feature configuration file, and the feature configuration file can be sent to the client through the ABT platform. In addition, the message platform is an information system that runs on a digital network and can support the push of multiple message types, including SMS, email, service numbers, and mini-programs, and can track the entire life cycle of messages. The message platform can also be used to configure features to form a feature configuration file, and the feature configuration file can be sent to the client through the message platform.

[0033] In the embodiment of the present application, the file format of the feature configuration file may include but is not limited to JSON and XML, etc., and the features can be flexibly defined in the feature configuration file according to different actual business needs. For example, assuming that an actual business may need a feature of clicking to enter the product details page, this feature can be defined as a "commercial details click feature", so as to facilitate the classification of feature data and improve the efficiency of using feature data.

[0034] In step S102, real-time feature data is collected based on the feature configuration file, and the real-time feature data is stored in the client database. In the embodiment of the present application, the aforementioned real-time feature data may include but is not limited to multi-dimensional feature data such as user behavior data, device information data, spatiotemporal data, etc., so as to provide rich contextual information. By utilizing the computing advantages of the client, higher-precision and real-time feature data collection is achieved, the reliability and applicability of the feature data are improved, and the collected real-time feature data can be stored in the client database, so that the collected real-time feature data can be uniformly stored and managed to ensure the integrity, consistency and traceability of the data.

[0035] In step S103, in response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server. This realizes the efficient transmission and distribution of real-time feature data, supports the rapid access and use of feature data in different application scenarios, and enables the server to determine the processing strategy based on the real-time feature data, so as to meet the rapid strategy output requirements of application scenarios with high real-time requirements such as product recommendation systems, real-time advertising push and personalized service customization, and improve the business processing efficiency of application scenarios with high real-time requirements.

[0036] The embodiment of the present application realizes the dynamic update of feature configuration without re-issuing the client application by receiving the feature configuration file issued by the server, thereby greatly shortening the iteration cycle, improving the development efficiency, and strengthening the simplicity and convenience of feature data collection. Then, based on the feature configuration file, real-time feature data is collected, so that the real-time collection of feature data can be realized through the client, the feature data transmission delay is reduced, the timeliness and accuracy of the feature data can be ensured, and the local computing power of the client can be used to reduce the computing burden of the server. In addition, the real-time feature data is stored in the client database, and the real-time feature data is uniformly stored and managed, ensuring the integrity, consistency and traceability of the data, reducing the number of transmissions of the real-time feature data on the network, effectively reducing the risk of data leakage, and enhancing data privacy protection. Then, in response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server, so that the server can determine the processing strategy based on the real-time feature data, thereby providing efficient data support and decision-making basis for actual business, and improving the quality and efficiency of business processing.

[0037] In general, this application can improve the real-time and accuracy of feature data, enhance the simplicity and convenience of feature data collection, and reduce the computing burden of the server.

[0038] In some embodiments, the feature configuration file may be parsed, and real-time feature data may be collected based on the parsing result. Figure 2 The following describes the real-time feature data collection process in detail. Figure 2 An exemplary flow chart 200 showing a method for collecting characteristic data in some other embodiments of the present application is shown in FIG. Figure 2 , the characteristic data collection method shown in the embodiment of the present application may include:

[0039] In step S201, the feature configuration file is parsed to obtain feature configuration information. In an embodiment of the present application, it can be first determined whether the feature configuration file has a historical download record based on the configuration file link in the feature configuration file. If so, the feature configuration file is not read and deleted; if not, the feature configuration file is read, and the feature version information and initial configuration information are read in the feature configuration file. It can be understood that each configuration information in the initial configuration information corresponds to a feature version information. Furthermore, the configuration information that does not conform to the local version information in the initial configuration information can be eliminated based on the feature version information and the local version information to obtain the feature configuration information. Among them, the aforementioned feature configuration information may include but is not limited to feature acquisition rules (such as feature acquisition frequency and feature acquisition restrictions, etc.), feature definition information and feature reporting rules (i.e., reporting methods of feature data). Thereby, effective version management of feature configuration can be achieved to ensure that each client can obtain the latest and correct feature definition.

[0040] In step S202, the local configuration information is updated based on the feature configuration information. In the embodiment of the present application, after obtaining the feature configuration information, the local feature configuration of the client needs to be initialized, that is, the service registration is performed using the feature configuration information including the feature collection rules, feature definition information and feature reporting rules, so as to achieve the purpose of updating the local configuration information.

[0041] In step S203, the preset hit event is intercepted to obtain the event original data. In an embodiment of the present application, the preset hit event may include the hit events pre-set in the client such as the buried event and the system behavior event. For the buried event, the buried event can be intercepted by an interceptor (such as an axios interceptor), so that the buried data (the buried data is business data, such as the business behavior data of clicking on a certain commodity, browsing a certain shop, etc., which can be regarded as the above-mentioned event original data) is intercepted when the buried event occurs on the client. For the system behavior event (such as network request, system method, etc.), the system behavior event can be intercepted by a hook program (HOOK), so as to obtain the event original data. Wherein, illustratively, the event original data can be the touch coordinate position of the user on the client device. It can be understood that the form of the event original data is diverse. In practical applications, the specific form of the event original data needs to be determined according to the actual application situation, and the present application does not make any restrictions in this regard.

[0042] In step S204, it is determined whether the event raw data meets the preset conditions. In the embodiment of the present application, the event raw data can be first converted into dictionary form data. For example, the event raw data can be converted into dictionary form data using the eval function. It is understandable that there are various ways to convert the event raw data into dictionary form data. In practical applications, it is necessary to determine the appropriate conversion method according to the actual application situation. The present application does not impose any restrictions in this regard.

[0043] Furthermore, it is possible to determine whether the dictionary parameters in the dictionary form data are within the preset value range. The WHERE syntax of SELECT in SQL can be used, specifically "SELECT a FROM global_rt_f_user_clk,teacherWHERE column_1IN('test1',\"test2\",'test3')OR column_2BETWEEN 1AND 20ANDcolumn_3NOT IN(1.2,5.3,9.4)". Among them, "column_1IN('test1',\"test2\",'test3')ORcolumn_2BETWEEN 1AND 20AND column_3NOT IN(1.2,5.3,9.4)" indicates the preset value range. If it is within the preset value range, it can be determined that the preset condition is met.

[0044] In step S205, if it meets the requirements, then the real-time feature data is collected from the original event data based on the feature configuration information. In the embodiment of the present application, if it meets the preset conditions, the SQL interface can be used to collect real-time feature data from the original event data based on the feature configuration information. Among them, the client can be divided into three types of caches, namely, memory cache, KV key value cache and SQLite disk cache, which is to cope with different business scenarios, such as some feature data only need to be cached in a single session, some feature data need to be cached in multiple sessions, and some feature data storage requires high performance, etc. Therefore, it adapts to different cache types and can uniformly use SQL syntax to perform operations such as data query and insertion. The efficiency of SQL syntax is that it defines a set of specifications for processing and querying data. By parsing SQL syntax, focusing on query as an example, it can realize complex WHERE condition judgments of data and judgments such as ordinary functions, aggregate functions, GROUP BY, HAVING, etc., and can realize the addition, deletion, modification, query and other operations of data in memory cache, KV key value cache and SQLite disk cache respectively. Thereby realizing the efficient query and processing capabilities of feature data in the original event data through the SQL interface, improving the dimension and ease of use of feature data collection.

[0045] Since the original data of events is messy and repetitive, the original data of events can be preprocessed in the client before real-time feature data collection is performed on the original data of events, which is beneficial to reduce the computing pressure on the server side. Specifically, preprocessing can include but is not limited to data filtering processing, data cleaning processing and data aggregation processing. Among them, data filtering processing is to filter the original data of events, such as event name matching, point-of-care parameter matching, etc.; in addition, data cleaning processing is to perform format standardization, outlier processing, data completion and other processing on the original data of events; furthermore, data aggregation processing is a processing method that merges multiple data into one aggregated data, such as requiring aggregate features such as the number of clicks per day and the number of visits per day of users.

[0046] In some embodiments, in the process of reporting real-time feature data, the real-time feature data can be first reported from the client to the middle layer by feature entrainment, and then reported from the middle layer to the server. Figure 3 The reporting process of real-time feature data is described in detail. Figure 3 An exemplary flow chart 300 showing a method for collecting characteristic data in some other embodiments of the present application is shown in FIG. Figure 3 , the characteristic data collection method shown in the embodiment of the present application may include:

[0047] In step S301, the real-time feature data is entrained in the reporting network request. In the embodiment of the present application, since different actual businesses have different requirements for feature data, the client can first determine the path whitelist of each reporting interface corresponding to each actual business in the reporting network request based on the real-time feature data and each business feature requirement corresponding to each actual business. It can be understood that the domain name and path (Path) in the path whitelist can be accessed without authentication. Then the path whitelist can be used to filter the features in the reporting process of real-time feature data. Exemplarily, assuming that the first actual business needs the feature of clicking to enter the product detail page, and the second actual business needs the feature of clicking to enter the store, then it is necessary to set a path whitelist matching the "feature of clicking to enter the product detail page" in the real-time feature data in the reporting interface corresponding to the first actual business, so that the first actual business can obtain the "feature of clicking to enter the product detail page" data through its corresponding reporting interface. Similarly, it is necessary to set a path whitelist matching the "feature of clicking to enter the store" in the real-time feature data in the reporting interface corresponding to the second actual business, so that the second actual business can obtain the "feature of clicking to enter the store" data through its corresponding reporting interface. In the embodiment of the present application, each business feature requirement corresponding to each actual business can be determined specifically based on the service_name passed as a parameter, for example, the service_name of the search business is search, the service_name of the recommendation business is rec, and so on. Furthermore, the real-time feature data can be enclosed in the reporting network request based on the path whitelist of each reporting interface, so that the corresponding real-time feature data can be reported to the middle layer of the server through each reporting interface. In the embodiment of the present application, the aforementioned middle layer is aggregated through multiple downstream API interfaces (i.e., reporting interfaces) of the server, wherein the downstream API interface can be, for example, a product recommendation downstream interface and a product center downstream interface, etc., so that each business feature requirement corresponding to each actual business can be passed to different downstream API interfaces according to the service_name.

[0048] In step S302, in response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server. In the embodiment of the present application, the real-time feature data in each reporting interface in the middle layer can be reported by means of a buried point channel and a dedicated channel. The setting of the middle layer can provide a reporting interface for the client application, simplify the development and maintenance of the client application, and can also be responsible for communicating with multiple actual businesses in the server, aggregating data, reducing the complexity of the client application, and can also provide functions such as load balancing, service discovery, caching, error handling and retry strategies to improve the overall performance and availability of the system. Thus, each actual business can obtain the required real-time feature data through its corresponding reporting interface, which is conducive to improving the analysis and decision-making efficiency of each actual business. Further, after the server receives the real-time feature data, the real-time feature data can be used for model reasoning and training, and the real-time feature data can also be used for business logic processing. For example, if the user does not pay after 15 minutes after placing an order, a PUSH message can be sent to remind the user.

[0049] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a device for feature data collection and corresponding embodiments.

[0050] Figure 4 FIG. 4 is a block diagram showing the hardware configuration of a device 400 for collecting feature data that can implement the feature data collection method of an embodiment of the present application. Figure 4 As shown, the device 400 for collecting characteristic data may include a processor 410 and a memory 420. Figure 4 In the device 400 for collecting feature data, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the device 400 for collecting feature data may also include Figure 4 The components shown in the figure are different from the common components. For example: fixed-point arithmetic units.

[0051] The device 400 for feature data collection may correspond to a computing device having various processing functions, such as functions for generating a neural network, training or learning a neural network, quantizing a floating-point neural network to a fixed-point neural network, or retraining a neural network. For example, the device 400 for feature data collection may be implemented as various types of devices, such as a personal computer (PC), a server device, a mobile device, etc.

[0052] The processor 410 controls all functions of the device 400 for feature data collection. For example, the processor 410 controls all functions of the device 400 for feature data collection by executing a program stored in the memory 420 on the device 400 for feature data collection. The processor 410 may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the device 400 for feature data collection. However, the present application is not limited thereto.

[0053] In some embodiments, the processor 410 may include an input / output (I / O) unit 411 and a computing unit 412. The I / O unit 411 may be used to receive various data, such as a feature configuration file issued by a server. Exemplarily, the computing unit 412 may be used to collect real-time feature data based on the feature configuration file received via the I / O unit 411, and store the real-time feature data in a client database, and then report the real-time feature data back to the server in response to a data transmission instruction issued by the server, so that the server can determine a processing strategy based on the real-time feature data. This real-time feature data may be output by the I / O unit 411, for example. The output data may be provided to the memory 420 for reading and use by other devices (not shown), or may be directly provided to other devices for use.

[0054] The memory 420 is hardware for storing various data processed in the device 400 for feature data collection. For example, the memory 420 can store processed data and data to be processed in the device 400 for feature data collection. The memory 420 can store data sets involved in the feature data collection method process that has been processed or to be processed by the processor 410, for example, a feature configuration file issued by a server, etc. In addition, the memory 420 can store applications, drivers, etc. to be driven by the device 400 for feature data collection. For example: the memory 420 can store various programs related to the feature data collection method to be executed by the processor 410. The memory 420 can be a DRAM, but the present application is not limited thereto. The memory 420 can include at least one of a volatile memory or a non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), etc. The volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), PRAM, MRAM, RRAM, ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 420 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (xD) card, caches, or a memory stick.

[0055] In summary, the specific functions implemented by the memory 420 and the processor 410 of the device 400 for feature data collection provided in the implementation manner of this specification can be explained in comparison with the aforementioned implementation manner in this specification, and can achieve the technical effects of the aforementioned implementation manner, and will not be repeated here.

[0056] In this embodiment, the processor 410 can be implemented in any suitable manner. For example, the processor 410 can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc.

[0057] It should also be understood that any module, unit, component, server, computer, terminal or device that executes instructions exemplified herein may include or otherwise access computer-readable media, such as storage media, computer storage media or data storage devices (removable and / or non-removable) such as disks, optical disks or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0058] The foregoing content can be better understood in accordance with the following terms:

[0059] Item A1, a feature data collection method, which includes: receiving a feature configuration file issued by a server; collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and in response to a data transmission instruction issued by the server, reporting the real-time feature data back to the server, so that the server can determine a processing strategy based on the real-time feature data.

[0060] Clause A2. A feature data collection method according to Clause A1, wherein the collection of real-time feature data based on the feature configuration file comprises: parsing the feature configuration file to obtain feature configuration information; wherein the feature configuration information includes feature collection rules, feature definition information and feature reporting rules; updating local configuration information based on the feature configuration information; intercepting preset hit events to obtain event raw data; determining whether the event raw data meets preset conditions; and if so, collecting the real-time feature data from the event raw data based on the feature configuration information.

[0061] Clause A3. A feature data collection method according to Clause A2, wherein the parsing of the feature configuration file comprises: determining whether the feature configuration file has a historical download record based on a configuration file link in the feature configuration file; if not, reading the feature configuration file to obtain feature version information and initial configuration information; and based on the feature version information and local version information, eliminating configuration information in the initial configuration information that does not conform to the local version information.

[0062] Clause A4. The feature data collection method according to Clause A2, wherein the preset hit event includes a tracking event and a system behavior event; wherein intercepting the preset hit event includes: intercepting the tracking event through an interceptor; and / or intercepting the system behavior event through a hook program.

[0063] Clause A5. A feature data collection method according to clause A2, wherein the step of determining whether the original event data meets the preset conditions comprises: converting the original event data into dictionary form data; determining whether a dictionary parameter in the dictionary form data is within a preset value range; and if so, determining that the preset conditions are met.

[0064] Clause A6. A feature data collection method according to clause A1, wherein, before reporting the real-time feature data back to the server in response to the data transmission instruction issued by the server, the method further comprises: enclosing the real-time feature data in the reporting network request.

[0065] Clause A7. A characteristic data collection method according to clause A6, wherein the step of including the real-time characteristic data in the network reporting request comprises: determining a path whitelist for each reporting interface corresponding to each actual service in the network reporting request based on the real-time characteristic data and each service characteristic requirement corresponding to each actual service; and including the real-time characteristic data in the network reporting request based on the path whitelist for each reporting interface.

[0066] Clause A8. According to the feature data collection method described in Clause A1, receiving the feature configuration file sent by the server includes: receiving the feature configuration file sent by the server through a configuration sending platform.

[0067] Item A9, a device for feature data collection, comprising: a memory; and at least one processor, configured to: receive a feature configuration file issued by a server; collect real-time feature data based on the feature configuration file, and store the real-time feature data in a client database; and in response to a data transmission instruction issued by the server, report the real-time feature data back to the server, so that the server can determine a processing strategy based on the real-time feature data.

[0068] Item A10, a non-transitory machine-readable medium having program code stored thereon for collecting feature data, wherein when the program code is executed by at least one processor, the code guides the execution operation of the at least one processor, and the program code includes: receiving a feature configuration file issued by a server; collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and in response to a data transmission instruction issued by the server, reporting the real-time feature data back to the server, so that the server can determine a processing strategy based on the real-time feature data.

[0069] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for collecting characteristic data, characterized in that: include: Receive the feature configuration file sent by the server; Collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; as well as In response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server, so that the server can determine a processing strategy based on the real-time feature data.

2. The characteristic data collection method according to claim 1, characterized in that: The collecting of real-time feature data based on the feature configuration file comprises: Parsing the feature configuration file to obtain feature configuration information; wherein the feature configuration information includes feature collection rules, feature definition information and feature reporting rules; Updating local configuration information based on the characteristic configuration information; Intercept the preset hit event and obtain the original data of the event; Determine whether the original event data meets the preset conditions; and If so, the real-time feature data is collected from the event raw data based on the feature configuration information.

3. The characteristic data collection method according to claim 2, characterized in that: The parsing of the feature configuration file comprises: Determining whether the feature configuration file has a historical download record based on a configuration file link in the feature configuration file; If not, read the feature configuration file to obtain feature version information and initial configuration information; and Based on the feature version information and the local version information, configuration information in the initial configuration information that does not conform to the local version information is removed.

4. The characteristic data collection method according to claim 2, characterized in that: The preset hit event includes a tracking point event and a system behavior event; wherein, intercepting the preset hit event includes: Intercepting the embedding event by an interceptor; and / or The system behavior event is intercepted by a hook procedure.

5. The characteristic data collection method according to claim 2, characterized in that: The determining whether the original event data meets the preset conditions includes: Converting the event raw data into dictionary form data; Determining whether a dictionary parameter in the dictionary form data is within a preset value range; and If so, it is determined that the preset condition is met.

6. The characteristic data collection method according to claim 1, characterized in that: Before reporting the real-time feature data back to the server in response to the data transmission instruction issued by the server, the method further includes: The real-time feature data is entrained in the reporting network request.

7. The characteristic data collection method according to claim 6, characterized in that: The step of entraining the real-time characteristic data in the reporting network request comprises: Determine, based on the real-time characteristic data and each service characteristic requirement corresponding to each actual service, a path whitelist of each reporting interface corresponding to each actual service in the reporting network request; and The real-time feature data is entrained in the reporting network request based on the path whitelist of each reporting interface.

8. The characteristic data collection method according to claim 1, characterized in that: The feature configuration file sent by the receiving server includes: Receive the feature configuration file sent by the server through a configuration sending platform.

9. A device for collecting characteristic data, characterized in that: include: Memory; as well as at least one processor configured to: Receive the feature configuration file sent by the server; Collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; and In response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server, so that the server can determine a processing strategy based on the real-time feature data.

10. A non-transitory machine-readable medium having stored thereon a program code for feature data collection, wherein when the program code is executed by at least one processor, the code guides the execution operation of the at least one processor, the program code comprising: Receive the feature configuration file sent by the server; Collecting real-time feature data based on the feature configuration file, and storing the real-time feature data in a client database; as well as In response to the data transmission instruction issued by the server, the real-time feature data is reported and fed back to the server, so that the server can determine a processing strategy based on the real-time feature data.

Images