Data stream-based burying point visualization verification method and device and readable storage medium thereof

Abstract

The invention provides a burying point visualization verification method and device based on data flow and a readable storage medium. User APP interaction data is obtained through burying point collection service, compressed and encoded and then transmitted to the collection service, abnormal data is analyzed and filtered, attributes such as timestamps and geographic information are added, and then the data is forwarded to Kafka; performing device ID strict matching filtering on the data based on a broadcast state by using streaming service, and writing the data meeting conditions into a database; multi-dimensional screening of time, equipment ID, event types and the like is supported through a visual interface, and real-time preview and verification of burying point data are achieved. The problems that in the prior art, buried point data cannot be previewed, the real-time performance is poor, and storage is inaccurate are solved, data integrity verification, second-level delay preview and accurate filtering are achieved through data flow full-link optimization, and buried point development efficiency and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer software technology, and specifically to the collection, processing, verification and visualization technology of buried point data, and in particular to a method, device and readable storage medium for buried point visualization verification based on data flow. Background Art

[0002] In existing technology, when users interact with an app, tracking data is reported by a collection service, processed, and forwarded to Kafka. A streaming service then reads the configuration file, consumes the data, and writes it to a MySQL database. Finally, the data is displayed through microservices. The specific process includes: tracking data is reported and processed by the collection service, the processed data is forwarded to Kafka, the streaming service consumes the data and writes it to the database, and finally, the microservice implements page display and filtering functions.

[0003] However, the existing technology has significant defects: 1. Data is not visualized: Tracking data lacks real-time preview capabilities during collection, processing, and storage, making it difficult for the business side to quickly verify the effectiveness of tracking.

[0004] 2. Inability to monitor in real time: The delay from point reporting to data display is high (minutes or even longer), resulting in delayed anomaly detection and the inability to detect data loss or format errors in a timely manner.

[0005] 3. Insufficient data accuracy: Existing solutions do not provide a dynamic filtering mechanism, cannot accurately locate buried data for specific devices or events, and lack integrity verification of private attributes.

[0006] Therefore, there is an urgent need for a data stream-based point-of-sight visualization verification method, device and readable storage medium thereof to solve the problems existing in the existing technology. Summary of the Invention

[0007] The embodiments of the present invention provide a data stream-based point-of-sale visualization verification method, device and readable storage medium thereof, which address the problems of the current technology in which the collection and processing processes of point-of-sale data lack real-time visualization, dynamic filtering and integrity verification mechanisms, resulting in high risk of data loss, delayed anomaly detection and low verification efficiency.

[0008] The core technology of this invention is mainly to collect, compress, encode, stream and filter the buried data based on broadcast JOIN, and combine it with real-time visual screening to achieve the integrity verification, real-time preview and accurate storage of the buried data based on data stream visualization verification method.

[0009] In a first aspect, the present invention provides a data flow-based point tracking visualization verification method, the method comprising the following steps: Collect user app interaction data through tracking and collection services. The interaction data includes public attributes and private attributes. Public attributes are located in the primary data structure, while private attributes are located in the secondary data structure, properties. Compress and encode the collected data and transmit it to the collection service; The collection service parses and processes the data, filters out abnormal data, adds timestamps, geographic information, and personalized attributes, and then forwards it to Kafka. Use streaming services to consume data from Kafka in real time, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID; The filtered data is written into the database, and a multi-dimensional filtering function is provided through a visual interface to achieve real-time preview and verification of the buried data.

[0010] Furthermore, the data is compressed and encoded, including: compressing the data using gzip, encoding the compressed data using Base64, and encoding the URL.

[0011] Furthermore, the collection service parses and processes the data, including: decoding and formatting the encoded data layer by layer, discarding non-compliant data and recording logs; obtaining the timestamp of the server's time zone and adding it to the data, parsing the IP through the IP address library to obtain geographic information and adding it to the data.

[0012] Furthermore, streaming services are used to consume data from Kafka in real time, and the data is filtered by device ID based on the broadcast status, including: configuring the device ID through the front-end interface and sending it to the broadcast stream, using a strict matching mode to filter the device ID in the buried data, and directly discarding data that does not meet the rules.

[0013] Furthermore, the streaming service uses the Flink or Spark Streaming engine to implement JOIN processing of low-throughput device filtering data streams and high-throughput embedded data streams through broadcast status; the Kafka tool class is initialized in singleton mode, sets the number of data sending retries, and configures device filtering rules.

[0014] Furthermore, the visual interface supports filtering based on time, platform, device ID, event type and attributes, and the returned data contains structured information such as event name, time, platform source and attribute definition.

[0015] Furthermore, the filtered data is written into the database, including: using JdbcSink to write into the database, setting a write retry mechanism and batch submission parameters.

[0016] In a second aspect, the present invention provides a data stream-based point tracking visualization verification device, comprising: The tracking collection module is used to collect user app interaction data through the tracking collection service. The interaction data includes public attributes and private attributes. Public attributes are located in the primary data structure, and private attributes are located in the secondary data structure properties. Compression module, used to compress and encode the collected data and transmit it to the collection service; The parsing module is used by the collection service to parse and process data, filter abnormal data, add timestamps, geographic information and personalized attributes, and forward it to Kafka; The streaming processing module is used to consume data from Kafka in real time using the streaming service, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID; Database writing module, used to write filtered data into the database; The visualization display module is used to provide multi-dimensional screening functions through a visual interface to achieve real-time preview and verification of tracking data.

[0017] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned data flow-based point visualization verification method.

[0018] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the above-mentioned data flow-based point visualization verification method.

[0019] The main contributions and innovations of the present invention are as follows: 1. Realize real-time preview and dynamic verification of tracking data By integrating the Flink stream computing engine with Kafka and adopting the broadcast state JOIN mechanism to process data streams, data latency is controlled to seconds. This solves the problem of real-time preview of tracking data in existing technologies and supports instant verification of the reporting status of interactive behavior tracking during the development and testing phases.

[0020] 2. Improve data accuracy and storage reliability By configuring the device ID on the front end and combining it with strict matching filtering rules, data from non-target devices is discarded in the streaming service, ensuring that only the data from the designated device is stored in the database. At the same time, the collection service performs three-level parsing and verification on the data (gzip decompression → Base64 decoding → format verification), discarding abnormal data and issuing an alarm, thus solving the problem of accurate data storage in existing technologies.

[0021] 3. Ensure data integrity and traceability The visual interface supports filtering by event name and displays all sub-attributes, facilitating manual verification of field integrity. The collection service adds server time zone timestamps and IP geographic information to avoid client-side time tampering and data traceability difficulties, resolving the defect in existing technologies where buried data cannot be previewed.

[0022] 4. Optimize system performance and resource utilization The Kafka tool class uses singleton mode initialization, which has been measured to reduce GC pressure by 85% and improve system stability. Before data transmission, it uses gzip compression (compression rate of approximately 22%) and Base64 encoding to reduce network transmission load, which is more efficient than existing technologies.

[0023] 5. Reduce the rate of missed reports and misjudgment of buried points Through real-time filtering of streaming services and a database write retry mechanism, combined with precise screening in a visual interface, the false positive rate of 2,893 tracking events was measured to be 0, ensuring that no tracking data was missed and resolving the issue of inaccurate verification caused by data loss in existing technologies.

[0024] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 This is a flow chart of a data flow-based point tracking visualization verification method according to an embodiment of the present invention; Figure 2 is a flow chart of compressing and encoding collected data according to an embodiment of the present invention; Figure 3 is a schematic diagram of obtaining a unique identifier of a device on a front-end page according to an embodiment of the present invention; Figure 4 is a schematic diagram of adding a device ID through a front-end page according to an embodiment of the present invention; Figure 5 is a flow chart of consuming data from Kafka in real time using a streaming service according to an embodiment of the present invention; Figure 6 is a schematic diagram of a visualization interface according to an embodiment of the present invention; Figure 7 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0027] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] The existing technology has the defects that the buried data cannot be previewed, cannot be previewed in real time, and cannot accurately store the specified data.

[0029] Based on this, the present invention solves the problems existing in the existing technology through data stream preprocessing (compression, encoding and secure transmission), dynamic filtering based on Flink broadcast state, quasi-real-time visual verification and high-availability architecture design.

[0030] Example 1 The present invention aims to propose a method for visual verification of buried points based on data flow, specifically, referring to Figure 1 , the method comprises the following steps: Step 1: Collect user app interaction data through tracking and collection services; In this embodiment, the buried point data is defined as follows in Table 1: Table 1

[0031] The raw data example of APP interaction row data is as follows: { "event": "$AppStart", "distinct_id": "5544635", "time": 1750082166639, "project": "production", "$device_id": "5FBA0693-3BA4-4458-91F2-1CF1B9EBE3B0", "$manufacturer": "Apple", "$os": "iOS", "$os_version": "17.4.1", "$model": "iPad13,8", "$app_version": "8.2.20", "$ip": "123.118.208.78", "properties": { "event_duration": 100 } } It can be seen that public attributes are attribute data that must be carried in all point data reports and are stored in the first-level data structure; private attributes are personalized attribute data for different events or attribute data added by the server and are stored in the properties second-level data structure.

[0032] Step 2: Compress and encode the collected data and transmit it to the collection service: 1. Gzip compress the data: Size before compression: 303 bytes; Compressed size: 236 bytes; Result: Compressed data is used for transmission.

[0033] 2. B64 encode the request data: Use Base64.encodeBase64 to convert binary data into ASCII characters (AZ, az, 0-9, +, / ) to avoid incorrect processing during transmission and ensure secure data transmission.

[0034] 3. Encode the url: Use URLEncoder.encode to convert special characters into a secure transmission format consisting of a % followed by two hexadecimal digits (e.g., space → %20). Decoding reverses the process (%20 → space). Encoding ensures URL transmission reliability, while decoding restores the original data to its original form.

[0035] Final uploaded data (gzip compressed + Base64 encoded tracking data example): H4sIAAAAAAAAE+2da3PTxhrHv4uGlwmzu1qtVnlHC8zwqsyh0 / Pi9IxGtpVYg235SHLSDMOMOZCThBASLim3AOW0QFtuBUoJEIbv0kay84qv0Eey44uwHdsxsZ1oJsN4p NVenv3 / 9qbnEf86JThGWhfGsCwhIipMRAqRRwTVsbT4SdVICGOES4rEFZmII4I+qWccYUzQsllV / y6uZx3DzKiWHjethABPjadydlKtz5EykZERIWHYjpGJO0GeAuEiZZI Iz2QtM6tbjqHbwtgpIWfrlqrFjJThTKspKC4FiSkkO2DCfcH46gT8DhJNGlnVdjQnB9ch+4xppbWUWqqYAXVE5XSQZgJqI2ysnS8sPfGuLQmVG9Mp / 4b75EHh6l2 / CDtu 6XpGnTISTlIYo5jCNT / DSd2yoZ2Qlh8kB0UEaWNaQtXicd22q8XZOTurZxJ6AnJwkmrKjJ+E3+NGSq+mSZsJPaU601m / ZG9+wVt965ds2JDOsh01oU0LY+NaytbhapDYb; Step 3: The collection service parses and processes the data, filters out abnormal data, adds timestamps, geographic information, and personalized attributes, and then forwards it to Kafka: In this embodiment, if Figure 2 As shown in the figure, data is parsed mainly through URL decoding → Base64 decoding → gzip decompression → public attribute verification layer by layer, filtering out abnormally formatted embedded data to ensure that the data structure entering the downstream (such as Kafka) is compliant and the content is complete.

[0036] in, Figure 2 The logic is as follows Table 2: Table 2

[0037] In this embodiment, the data is processed as follows: 1. Add the timestamp of the server's time zone when the data is received from the service to solve the problem that the client can modify the time and avoid reporting the wrong time; 2. Obtain the IP address from the request header, process IPV4 and IPV6 requests through the IP address library, parse the IP address, and add detailed geographic information such as country and province to the original tracking data; 3. Increase personalized needs, such as user login status judgment, platform affiliation, etc.

[0038] The data after adding attributes is as follows: { "event": "$AppStart", "distinct_id": "5544635", "time": 1750082166639, "project": "production", "$device_id": "5FBA0693-3BA4-4458-91F2-1CF1B9EBE3B0", "$manufacturer": "Apple", "$os": "iOS", "$os_version": "17.4.1", "$model": "iPad13,8", "$app_version": "8.2.20", "$ip": "123.118.208.78", "properties": { "event_duration": 100, "$city": "Beijing", "$country": "China", "$province": "Beijing", "recv_time": 1750082175166, } } In this embodiment, sending data to Kafka is as follows: 1. Initialize the Kafka tool class using the singleton mode: Performance improvement: Avoid frequent creation and destruction of objects (measured reduction of 85% GC pressure); Memory saving: only one instance exists in the system memory; Globally unique access point: uniformly obtain instances through static methods; Initialization control: support lazy loading.

[0039] 2. Set the number of retries to ensure successful data transmission.

[0040] Step 4: Use the streaming service to consume data from Kafka in real time, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID.

[0041] In this embodiment, device ID filtering is achieved by adding filtering devices: 1. Such as Figure 3 As shown, through the front-end page, enter the account to automatically obtain the device's unique identifier, that is, the device number (ID); 2. Such as Figure 4 As shown, add the device ID through the front-end page and automatically send it to the broadcast stream.

[0042] In this embodiment, the filtering rules for device ID filtering are as follows: 1. Use strict matching mode. The device ID in the tracking data must be consistent with the device ID to be filtered to ensure the validity of the data. 2. Perform filtering operations when the stream service consumes Kafka data; 3. Data that does not meet the filtering rules is discarded directly, and only valid embedded data is retained.

[0043] Preferably, the streaming service is as follows: Goal: Streaming data and delivering real-time results. Supported open source engines: Flink / SparkStreaming; Using Flink: integrated streaming and batch architecture; state management; efficient fault tolerance mechanism; integration with Kafka; broadcast-based dual-stream JOIN; integration with JDBC.

[0044] Preferably, a dual-stream JOIN based on broadcasting: 1. Broadcast State is a special state type provided by Flink that allows data from one stream to be broadcast to all parallel tasks and processed in conjunction with data from another stream. 2. Broadcast stream: low-throughput dimensional data (device filtering data stream); 3. Main data stream: high-throughput factual data (buried data stream).

[0045] In this embodiment, JdbcSink is used on the data writing side, and key configurations such as retry and size during writing are set to ensure that data is stored normally.

[0046] like Figure 5 As shown in the figure, this step mainly uses Flink's Broadcast State feature to achieve real-time association between the "high-throughput tracking flow" and the "low-throughput device rule flow", solving the problem that "static configuration files cannot dynamically update filtering rules" in traditional methods.

[0047] in, Figure 5 The logic is as follows Table 3: Table 3

[0048] This step uses Flink broadcast streams to transmit filtering rules in real time, allowing embedded data to flow like water through a screen. Only the "water droplets" (target device data) that meet the rules will flow into the database, supporting the core advantage of "precise warehousing" of this invention.

[0049] Step 5: Write the filtered data into the database and provide multi-dimensional filtering functions through the visual interface to achieve real-time preview and verification of the embedded data, such as Figure 6 shown.

[0050] In this embodiment, the goal of data visualization is to increase the visual display of filtered data, facilitate data inspection and verification on the business side, support the filtering of filtered devices, and facilitate viewing the data of specified devices. The specific implementation plan is as follows: On the client side: 1. Define unified request parameters: { "startTime": "", "endTime": "", "appFrom": "", "event": "home_page_uppertab_impression", "deviceId": ["AB727BA3-2F71-46F6-86F2-02AEBB383D86"], "attrFilter": "", "pageNum": 1, "pageSize": 10 } 2. Support page filtering: date, platform, device number, event, attribute; 3. Support paging.

[0051] On the server side: 1. Define standard structure data return: { "success": true, "code": null, "message": null, "data": { "data": { "youthmodel_is_open": false, "$model": "iPhone12,1", "$os": "iOS", "model_type": "Mobile phone", "project": "Yingqu", "user_vip_status": 2, "brand_name": "iPhone", "$ip": "115.193.78.247", "user_style": "Student", "$app_version": "8.2.20", "user_stage": "Office worker", "tab_name": "Dubbing", "time": "2025-06-19 09:33:36", "event": "home_page_uppertab_impression", "$is_login_id": true, "user_ability_level": "1" } , "title": { "event": "Event name", "project": "Platform source", "time": "Time" }, "totalCount": 5 }<000027​​​​​​​​​​​The tracking collection module is used to collect user app interaction data through the tracking collection service. The interaction data includes public attributes and private attributes. Public attributes are located in the primary data structure, and private attributes are located in the secondary data structure properties. Compression module, used to compress and encode the collected data and transmit it to the collection service; The parsing module is used by the collection service to parse and process data, filter abnormal data, add timestamps, geographic information and personalized attributes, and forward it to Kafka; The streaming processing module is used to consume data from Kafka in real time using the streaming service, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID; Database writing module, used to write filtered data into the database; The visualization display module is used to provide multi-dimensional screening functions through a visual interface to achieve real-time preview and verification of tracking data.

[0053] Example 3 This embodiment also provides an electronic device, referring to Figure 7 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0054] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0055] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0056] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0057] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the data flow-based point-of-sale visualization verification methods in the above embodiments.

[0058] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0059] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0060] The input / output device 408 is used to input or output information.

[0061] Example 4 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes a data flow-based point visualization verification method according to embodiment one.

[0062] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0063] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0064] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0065] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A data flow-based point-of-sale visualization verification method, characterized in that: The following steps are involved: Collect user APP interaction data through the tracking collection service. The interaction data includes public attributes and private attributes, where public attributes are located in the primary data structure and private attributes are located in the secondary data structure properties; Compress and encode the collected data and transmit it to the collection service; The collection service parses and processes the data, filters out abnormal data, adds timestamps, geographic information, and personalized attributes, and then forwards it to Kafka. Use streaming services to consume data from Kafka in real time, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID; The filtered data is written into the database, and a multi-dimensional filtering function is provided through a visual interface to achieve real-time preview and verification of the buried data.

2. A data flow-based point-of-sale visualization verification method according to claim 1, characterized in that: Compress and encode the data, including: compressing the data with gzip, encoding the compressed data with Base64, and encoding the URL.

3. A data flow-based point-of-sale visualization verification method according to claim 1, characterized in that: The collection service parses and processes the data, including: decoding and formatting the encoded data layer by layer, discarding non-compliant data and recording logs; obtaining the timestamp of the server's time zone and adding it to the data, parsing the IP address through the IP address library to obtain geographic information and adding it to the data.

4. A data flow-based point-of-sale visualization verification method according to claim 1, characterized in that: Utilize streaming services to consume data from Kafka in real time and filter the data by device ID based on the broadcast status. This includes: configuring the device ID through the front-end interface and sending it to the broadcast stream, using a strict matching mode to filter the device ID in the embedded data, and directly discarding data that does not meet the rules.

5. The data flow-based point-of-sale visualization verification method according to claim 1, characterized in that: The streaming service uses the Flink or Spark Streaming engine, and implements JOIN processing of low-throughput device filtering data streams and high-throughput embedded data streams through broadcast status; the Kafka tool class is initialized in singleton mode, sets the number of data sending retries, and configures device filtering rules.

6. A data flow-based point-of-sale visualization verification method according to claim 1, characterized in that: The visual interface supports filtering based on time, platform, device ID, event type and attributes, and the returned data includes structured information such as event name, time, platform source and attribute definition.

7. A data flow-based point-of-sale visualization verification method according to any one of claims 1 to 6, characterized in that: Write the filtered data into the database, including: using JdbcSink to write to the database, setting the write retry mechanism and batch submission parameters.

8. A data flow-based point-of-use visualization verification device, characterized in that: include: The tracking point collection module is used to collect user APP interaction data through the tracking point collection service. The interaction data includes public attributes and private attributes, where public attributes are located in the primary data structure and private attributes are located in the secondary data structure properties; Compression module, used to compress and encode the collected data and transmit it to the collection service; The parsing module is used by the collection service to parse and process data, filter abnormal data, add timestamps, geographic information and personalized attributes, and forward it to Kafka; The streaming processing module is used to consume data from Kafka in real time using the streaming service, filter the data by device ID based on the broadcast status, and only retain data that matches the preset device ID; Database writing module, used to write filtered data into the database; The visualization display module is used to provide multi-dimensional screening functions through a visual interface to achieve real-time preview and verification of tracking data.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the data flow-based point-of-sale visualization verification method described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: A computer program is stored in the readable storage medium, and the computer program includes a program code for controlling a process to execute a process, and the process includes the data flow-based point-of-sale visualization verification method according to any one of claims 1 to 7.

Images