User behavior data acquisition method and device
By introducing message queues and preset acquisition conditions in the buried point technology, flexible buried point data acquisition is achieved, solving the problem of excessive system resource consumption and inability to meet application requirements, and achieving efficient and low resource consumption of user behavior data collection.
Patent Information
- Application Number
- CN202411527054.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-14
AI Technical Summary
The existing buried point technology has caused excessive system resource consumption during data acquisition and cannot meet the flexibility of application requirements.
Through message queues and preset acquisition conditions, the buried point operations are deployed and managed to achieve flexible buried point data acquisition. The specific steps include extracting user behavior data in the message queue and generating buried point data when the data meets preset collection conditions.
It realizes flexible and low resource consumption user behavior data collection, solving the problem of excessive system resource consumption and inability to meet application needs.
Smart Images
Figure CN119441617A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for collecting user behavior data. Background Art
[0002] With the rapid development of the Internet and mobile applications, understanding user behavior is crucial to improving user experience and product optimization. Data tracking technology captures data for specific user behaviors or events, based on which developers can analyze user behavior and collect data such as user preferences and behavioral characteristics. This data can be used to provide personalized content recommendations and services to enhance user experience.
[0003] The tracking technology requires setting up a tracking object, that is, adding code at the location where user behavior data needs to be detected to collect information when user behavior occurs, such as the number of visits, number of visitors, length of stay, number of page views, bounce rate, and other information generated with user operation behavior.
[0004] In order to obtain richer data, it is often necessary to deploy a large number of tracking objects, which affects system performance. As the number of users continues to increase, tracking technology consumes more system resources. Due to changes in business needs, resource allocation and other factors, the existing tracking object settings may not match the actual application scenario, affecting the directionality of the collected data. Summary of the invention
[0005] In order to overcome the problems existing in the related technologies, the present invention provides a user behavior data collection method and device. Through the message queue and the preset collection conditions, the point-of-sale operation is deployed and managed, and flexible point-of-sale data collection is realized, which solves the problem of excessive system resource consumption and failure to meet application requirements during the point-of-sale data collection process.
[0006] According to a first aspect of an embodiment of the present disclosure, a method for collecting user behavior data is provided, comprising: Extract user behavior data from the message queue; When the user behavior data meets the preset collection conditions, the buried data is generated according to the user behavior data.
[0007] Furthermore, the collection conditions include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interface that needs to be collected, the user behavior pattern that needs to be collected, and the business requirement type that needs to be collected. The method further comprises: Configure the acquisition conditions.
[0008] Furthermore, the message queue includes at least one piece of user behavior data, and the method further includes: Receive business requests; In the process of executing service for the business request, the user behavior data of the business request is saved in the message queue.
[0009] Furthermore, the user behavior data includes interface information of corresponding business requests.
[0010] Furthermore, in the process of performing service on the business request, before the step of saving the user behavior data of the business request in the message queue, the method further includes: Performing a security check on the service request, wherein the security check includes parameter check and / or user identity check; When the security check passes, the service for the business request is started.
[0011] Furthermore, after the step of performing security verification on the service request, the following step is further included: If the security check fails, the service request is rejected.
[0012] Furthermore, the method further comprises: Configure the parameters of the security check.
[0013] According to a second aspect of an embodiment of the present disclosure, there is provided a user behavior data collection device, comprising: Data extraction module, used to extract user behavior data in the message queue; The data collection module is used to generate buried data according to the user behavior data when the user behavior data meets the preset collection conditions.
[0014] Furthermore, the message queue includes at least one piece of user behavior data, and the device further includes: A request receiving module, used for receiving a service request; The message queue management module is used to save the user behavior data of the business request into the message queue during the process of performing service on the business request.
[0015] Furthermore, the device also includes: The request management module is used to perform security verification on the business request, the security verification includes parameter verification and / or user identity verification, and start executing the service for the business request if the security verification passes, and reject the business request if the security verification fails.
[0016] Furthermore, the device also includes: The acquisition condition management module is used to configure the acquisition conditions, which include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interfaces that need to be collected, the user behavior patterns that need to be collected, and the types of business requirements that need to be collected; The verification parameter management module is used to configure the parameters of the security verification.
[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: extracting user behavior data from a message queue, and generating point-of-sale data based on the user behavior data when the user behavior data meets the preset collection conditions. User behavior data is managed by caching through a message queue, and point-of-sale data is only generated for user behavior data that meets the preset collection conditions. Flexible and low-resource user behavior data collection is achieved through message queues and collection conditions, solving the problem of excessive system resource consumption and failure to meet application requirements during point-of-sale data collection.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0020] Figure 1 The figure is a flow chart of a method for collecting user data according to an exemplary embodiment.
[0021] Figure 2 The figure is a flow chart of another method for collecting user data according to an exemplary embodiment.
[0022] Figure 3 The figure is a flow chart of another method for collecting user data according to an exemplary embodiment.
[0023] Figure 4 The figure is a flow chart of another method for collecting user data according to an exemplary embodiment.
[0024] Figure 5 The figure is a flow chart of another method for collecting user data according to an exemplary embodiment.
[0025] Figure 6The figure is a block diagram of a user data collection device according to an exemplary embodiment.
[0026] Figure 7 It is a block diagram of yet another user data collection device according to an exemplary embodiment.
[0027] Figure 8 It is a block diagram of yet another user data collection device according to an exemplary embodiment.
[0028] Fig. 9 It is a block diagram of yet another user data collection device according to an exemplary embodiment. DETAILED DESCRIPTION
[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0030] In order to obtain richer data, it is often necessary to deploy a large number of tracking objects, which affects system performance. As the number of users continues to increase, tracking technology consumes more system resources. Due to changes in business needs, resource allocation and other factors, the existing tracking object settings may not match the actual application scenario, affecting the directionality of the collected data.
[0031] In order to solve the above problems, the embodiments of the present disclosure provide a method and device for collecting user behavior data. Through message queues and preset collection conditions, the tracking point operation is deployed and managed, which realizes flexible tracking point data collection and solves the problem of excessive system resource consumption and failure to meet application requirements during tracking point data collection.
[0032] An exemplary embodiment of the present disclosure provides a method for collecting user behavior data. The process of using the method to collect point data is as follows: Figure 1 As shown, including: Step 101: extract user behavior data in the message queue.
[0033] In this step, the user behavior data is stored in a message queue (MQ).
[0034] According to some implementations, at least one user behavior data is stored in the message queue, each user behavior data corresponds to a service request, and multiple user behavior data may correspond to one or more service requests.
[0035] According to some implementations, a message queue refers to a container for storing messages, which is essentially a queue for storing data. In a message queue, user behavior data collected based on business requests over a period of time is stored. Due to the use of a message queue, the process of collecting user behavior data and the subsequent process of collecting buried data can be performed asynchronously without interfering with each other.
[0036] According to some embodiments, the message queue may be RabbitMQ.
[0037] Step 102: When the user behavior data meets the preset collection conditions, generate buried data according to the user behavior data.
[0038] In the embodiment of the present invention, the acquisition conditions include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interfaces that need to be collected, the user behavior patterns that need to be collected, and the types of business requirements that need to be collected.
[0039] According to some implementations, the user behavior data carries the request interface parameters of the user initiating the service request. When the request interface parameters indicate that it is a service request interface that needs to be collected, the user behavior data is processed by embedding to generate embedding data.
[0040] According to some implementations, the user behavior data carries the request interface parameters of the user who initiated the service request. When the request interface parameters indicate that it is a service request interface that needs to be collected, it is further determined whether the recording frequency of the user behavior data meets the collection frequency condition (the collection frequency condition can be a frequency value. When the value is exceeded, it is determined that the collection frequency condition is not met, otherwise it is determined that the collection frequency condition is met) to ensure that data is not recorded excessively, the amount of duplicate data is increased, and system resources are wasted. After the user behavior data is confirmed and recorded, the system will mark the point-burying process as complete.
[0041] According to some implementations, after extracting the user behavior data, it can be stored in a database to ensure the accuracy and consistency of the data while also taking into account the security of the data and the protection of user privacy.
[0042] In an embodiment of the present invention, the collection conditions can be configured. The collection conditions can include multiple items, and the content of each item can be changed according to actual application needs, such as changing the business request interface and user behavior mode; items can also be added or deleted. By flexibly configuring the collection conditions, the scope of the buried data operation can be adjusted and controlled in a timely manner to better meet user needs and reduce system pressure.
[0043] After completing data tracking, the tracking data or data collection results can be fed back to the relevant system or user to confirm that the user's collected data has been successfully recorded and processed for subsequent monitoring, auditing, etc., to ensure the transparency and reliability of the data collection process.
[0044] An exemplary embodiment of the present invention also provides a method for collecting user behavior data. The process of adding user behavior data to a message queue using this method is as follows: Figure 2 As shown, including: Step 201: Receive a service request.
[0045] In this step, the business platform receives the business request and processes the business request, such as calling the corresponding service to respond or reject it. It can also analyze and provide information carried or related to the business request, such as the business request interface, user information, etc.
[0046] According to some implementations, the business platform may include a gateway, which may be a microservice gateway. For example, an API gateway, where multiple API interfaces for different services, different users, and different authentication requirements interact with users through the API gateway, making the service transparent to the caller and reducing the coupling between the client and the backend. At the same time, backend services are aggregated to save traffic, improve performance, and enhance user experience.
[0047] Step 202: In the process of executing service for the business request, the user behavior data of the business request is saved in the message queue.
[0048] According to some implementation modes, the user behavior data includes interface information of corresponding business requests.
[0049] In this step, the user behavior data is stored while the business platform provides services for business requests and executes business methods. Specifically, the user behavior data is saved in the message queue for the point-of-care data collection service to call. This enables the asynchronous processing of business request processing and the point-of-care data collection process, making service deployment and updating more flexible.
[0050] An exemplary embodiment of the present disclosure also provides a user data collection method, which performs security control on multiple users at the gateway level. The specific process is as follows Figure 3 As shown, including: Step 301: Receive a service request.
[0051] The implementation principle of this step is the same as that of step 201, and will not be repeated here.
[0052] Step 302: Perform security check on the service request.
[0053] In this step, the security verification includes parameter verification and / or user identity verification.
[0054] According to some implementations, the safety verification process includes: 1. Preliminary identification of received business requests.
[0055] 2. Perform parameter verification on the received business request.
[0056] This includes checking whether the business request contains all necessary parameters, whether the parameter type is correct, and whether the value is within the allowed range.
[0057] 3. After the parameter verification is passed, the identity of the user who issued the business request is verified to confirm the user identity.
[0058] Security verification, authentication, authorization and other operations can be implemented based on business requests at the gateway level, without having to be implemented separately at each service node that provides the service. The gateway manages the service nodes in a unified manner, and adding or deleting service nodes is done at the gateway, making it easier to adjust system capacity.
[0059] According to some embodiments, the gateway may be a microservice gateway, such as Zuul.
[0060] The unified management of the security levels of different services through the gateway improves the maintainability and scalability of the system, and also enhances the efficiency and security of data processing.
[0061] In the embodiment of the present invention, the parameters of the security check can also be configured. Specifically, the parameters of the security check can be configured according to user requirements, system environment, software and hardware configuration and other conditions.
[0062] After the security check is passed, step 303 and step 304 may be executed. It should be noted that there is no strict time sequence relationship between step 303 and step 304. During the execution of the service, the user behavior data may be sent to the message queue for storage.
[0063] Step 303: When the security check passes, start executing the service for the business request.
[0064] In this step, if the service request passes the security check, the service request is responded to and forwarded to the corresponding service.
[0065] Step 304: In the process of executing the service for the business request, the user behavior data of the business request is saved in the message queue.
[0066] The implementation principle of this step is the same as that of step 202, and will not be repeated here.
[0067] Step 305: If the security check fails, reject the service request.
[0068] In this step, if the security check fails, it is determined that the business request does not meet the platform security requirements and the corresponding service is refused.
[0069] An exemplary embodiment of the present disclosure also provides a method for collecting user behavior data. The process of collecting user behavior data using this method is as follows: Figure 4 As shown, including: Step 401: Receive a service request.
[0070] Step 402: Perform security verification on the service request.
[0071] In this step, the security verification includes parameter verification and / or user identity verification.
[0072] If the service request passes the security check, go to step 403 ; otherwise, go to step 405 .
[0073] When the security check passes, the service for the business request is started.
[0074] Step 403: When the security check passes, start executing the service for the business request.
[0075] Step 404: In the process of executing the service for the business request, the user behavior data of the business request is saved in the message queue.
[0076] When there is at least one piece of user behavior data in the message queue, the user behavior can be extracted and it can be determined whether data tracking needs to be performed, and the process proceeds to step 405 .
[0077] Step 405: If the security check fails, reject the service request.
[0078] Step 406: extract user behavior data from the message queue.
[0079] Step 407: When the user behavior data meets the preset collection conditions, generate buried data according to the user behavior data.
[0080] It should be noted that there is no strict time sequence relationship between the process of storing user behavior data in the message queue in step 403 and the process of performing data embedding and collecting embedding data in steps 406-407. Since the message queue is introduced to store and manage user behavior data, the processes shown in steps 403 and 406-407 can be performed asynchronously, decoupling the data collection system and the processing system, thereby improving the performance and responsiveness of the system.
[0081] An exemplary embodiment of the present disclosure also provides a method for collecting user behavior data. Taking the use of a gateway for security management as an example, the process of using the method to collect user behavior data through data embedding is as follows: Figure 5 As shown, including: Step 501: The user sends a service request.
[0082] In this step, the user sends a service request to the gateway through the request interface A, and carries the request interface information and user behavior data in the service request.
[0083] In the embodiment of the present invention, the gateway may be a microservice gateway.
[0084] Step 502: The gateway performs parameter verification.
[0085] In this step, the gateway performs parameter security verification on the service request. The gateway performs parameter verification on the received service request, including checking whether the service request contains all necessary parameters, whether the parameter type is correct, and whether the value is within the allowed range. After the parameter legitimacy verification passes, the gateway performs identity verification to confirm the identity of the user who sent the service request.
[0086] If the verification fails, a service request failure message is returned to the user and the service process is terminated.
[0087] Step 503: The gateway performs user identity verification.
[0088] In this step, the gateway verifies the user identity of the service request initiator.
[0089] If the verification fails, a service request failure message or an identity authentication failure message is returned to the user, terminating the service process.
[0090] Step 504: The gateway forwards the service request to the corresponding service.
[0091] In this step, according to the service pointed to by the business request, the business request is forwarded to a specific service to execute the service in response to the business request.
[0092] Step 505: After the service is executed, the gateway feeds back the service response to the user.
[0093] In this step, after the service is executed, the processing results and interface parameters are encapsulated into a business response and fed back to the user. This ensures the timeliness and accuracy of the response and provides clear feedback to the user.
[0094] Step 506: Add the user behavior data to the message queue.
[0095] In this step, it is asynchronously sent to the message queue (MQ) to wait for subsequent asynchronous processing. This step improves the performance and responsiveness of the system by decoupling data collection and processing.
[0096] Step 507: extract the user behavior data in the message queue and determine whether data embedding is required.
[0097] In this step, the user behavior data is analyzed to determine whether data embedding is required for the user behavior data.
[0098] The asynchronous service obtains user behavior data from MQ and stores it in the database. This step ensures the accuracy and consistency of the data while taking into account the security and privacy protection of the data. It determines whether the current business request meets the collection conditions to indicate whether data embedding is required.
[0099] This judgment is based on the business request interface, user behavior pattern or specific business needs. This step is part of the data collection strategy to ensure that only user behavior data that meets specific conditions is recorded.
[0100] When it is determined that the collection conditions are not met, the collection process for the user behavior data is terminated, and the tracking point analysis of other user behavior data in the message queue continues.
[0101] Step 508: Determine the frequency of buried points.
[0102] In this step, the frequency of tracking records is determined based on the business request interface to ensure that data is not excessively recorded.
[0103] Step 509: Execute data tracking to record user behavior data.
[0104] In this step, the asynchronous service performs data tracking, generates tracking events based on user behavior data, and encapsulates them into tracking data, such as generating user behavior collection messages. This step is the basis for user behavior analysis and business insights.
[0105] Step 510: After completing the tracking, the system provides feedback to the relevant system or user.
[0106] In this step, the system provides feedback to confirm that the data has been successfully recorded and processed. This step helps with monitoring and auditing, ensuring transparency and accountability of the data collection process.
[0107] An exemplary embodiment of the present disclosure also provides a user behavior data collection device, whose structure is as follows: Figure 6 As shown, including: The data extraction module 601 is used to extract the user behavior data in the message queue; The data collection module 602 is used to generate embedded data according to the user behavior data when the user behavior data meets the preset collection conditions.
[0108] Furthermore, the message queue includes at least one user behavior data, and the device Figure 7 As shown, it also includes: A request receiving module 603, used for receiving a service request; The message queue management module 604 is used to save the user behavior data of the business request into the message queue during the process of performing service on the business request.
[0109] Furthermore, the structure of the device is as follows Figure 8 As shown, it also includes: The request management module 605 is used to perform security verification on the business request, the security verification includes parameter verification and / or user identity verification, and start executing the service for the business request if the security verification passes, and reject the business request if the security verification fails.
[0110] Furthermore, the structure of the device is as follows Fig. 9 As shown, it also includes: The collection condition management module 606 is used to configure the collection conditions, which include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interfaces that need to be collected, the user behavior patterns that need to be collected, and the types of business requirements that need to be collected; The verification parameter management module 607 is used to configure the parameters of the security verification.
[0111] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here. The above device can be integrated into a microservice gateway, and the microservice gateway implements the corresponding functions.
[0112] The embodiments of the present disclosure provide a method and device for collecting user behavior data, extracting user behavior data from a message queue, and generating point-of-sale data based on the user behavior data when the user behavior data meets preset collection conditions. User behavior data is managed by caching through a message queue, and point-of-sale data is only generated for user behavior data that meets preset collection conditions. Flexible and low-resource user behavior data collection is achieved through message queues and collection conditions, solving the problem of excessive system resource consumption and failure to meet application requirements during point-of-sale data collection.
[0113] In pages or operations where users interact frequently. As the user base and data volume grow, traditional tracking methods may be difficult to expand to meet the needs of large-scale data collection.
[0114] Check the legitimacy of the protocol, the legitimacy of the parameters, and the validity of the data.
[0115] The distributed collection method based on gateways and message queues not only improves the maintainability and scalability of the system, but also enhances the efficiency and security of data processing. The gateway can add new service nodes to handle more user requests, while the message queue can expand the message processing capacity by adding queues and users. Compared with the traditional synchronous data collection method, the system based on gateways and message queues can realize asynchronous processing, reduce user waiting time, and improve system response speed and user experience. The gateway and message queue under the microservice architecture make the update and deployment of services more flexible and independent, reducing the risks that may be brought about by system updates in traditional technologies.
[0116] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0117] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A user behavior data collection method, characterized in that: include: Extract user behavior data from the message queue; When the user behavior data meets the preset collection conditions, the buried data is generated according to the user behavior data.
2. The user behavior data collection method according to claim 1, characterized in that: The acquisition conditions include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interface that needs to be collected, the user behavior pattern that needs to be collected, and the business requirement type that needs to be collected. The method further comprises: Configure the acquisition conditions.
3. The user behavior data collection method according to claim 1, characterized in that: The message queue includes at least one piece of user behavior data, and the method further includes: Receive business requests; In the process of executing service for the business request, the user behavior data of the business request is saved in the message queue.
4. According to claim 3 The user behavior data collection method is characterized in that: In the process of performing service on the business request, before the step of saving the user behavior data of the business request in the message queue, the step further includes: Performing a security check on the service request, wherein the security check includes parameter check and / or user identity check; When the security check passes, the service for the business request is started.
5. The user behavior data collection method according to claim 4, characterized in that: After the step of performing security verification on the service request, the method further includes: If the security check fails, the service request is rejected.
6. The user behavior data collection method according to claim 4, characterized in that: The method further comprises: Configure the parameters of the security check.
7. A user behavior data collection device, characterized in that: include: Data extraction module, used to extract user behavior data in the message queue; The data collection module is used to generate buried data according to the user behavior data when the user behavior data meets the preset collection conditions.
8. The user behavior data collection device according to claim 7, characterized in that: The message queue includes at least one piece of user behavior data, and the device further includes: A request receiving module, used for receiving a service request; The message queue management module is used to save the user behavior data of the business request into the message queue during the process of performing service on the business request.
9. The user behavior data collection method according to claim 8, characterized in that: The device also includes: The request management module is used to perform security verification on the business request, the security verification includes parameter verification and / or user identity verification, and start executing the service for the business request if the security verification passes, and reject the business request if the security verification fails.
10. The user behavior data collection device according to claim 7, characterized in that: The device also includes: The acquisition condition management module is used to configure the acquisition conditions, which include at least one or more of the following conditions: Business parameter conditions, collection frequency conditions, The service parameter condition includes at least one or more of the following parameters: The business request interfaces that need to be collected, the user behavior patterns that need to be collected, and the types of business requirements that need to be collected; The verification parameter management module is used to configure the parameters of the security verification.