Internet of vehicles V2X data desensitization method and device, and medium

Abstract

The invention discloses an Internet of Vehicles V2X data desensitization method and device and a medium, and relates to the technical field of Internet of Vehicles data security. The method comprises the following steps: acquiring multi-source data in a V2X scene by adopting a SeaTunnel native connector; based on a predefined rule base and a machine learning classification model, determining the sensitivity level of each field in the multi-source data; and adopting a desensitization plug-in to perform hierarchical desensitization on the field based on the sensitivity level of the field and the service scene to obtain desensitization data corresponding to the field. Therefore, identification and desensitization of the V2X data can be realized based on the SeaTunnel.

Description

Technical Field

[0001] This application relates to the field of vehicle-to-everything (V2X) data security technology, and in particular to a method, device and medium for desensitizing V2X data. Background Technology

[0002] With the widespread application of vehicle-to-everything (V2X) technology, massive amounts of data containing sensitive and private information (such as vehicle VIN codes, precise location information, vehicle owner identification, license plate numbers, etc.) are generated during the communication process of the Internet of Vehicles (V2X).

[0003] Existing technologies suffer from four major shortcomings: First, general-purpose data masking tools (such as the built-in data masking component in Flink CDC) fail to adapt to the multimodal data characteristics of V2X, making it difficult to simultaneously meet the data masking requirements of structured data (such as vehicle speed and location coordinates), semi-structured data (such as communication certificates), and unstructured data (such as image metadata); Second, traditional batch processing data masking modes (such as database encryption schemes) cannot meet the millisecond-level real-time interaction response requirements in V2X scenarios; Third, they lack intelligent sensitive data identification capabilities, rely excessively on manual rule definitions, and are unable to cope with the challenges of dynamic changes in V2X data fields.

[0004] SeaTunnel, as a distributed data integration tool, supports streaming processing architectures based on the Flink / Zeta engine and custom Transform development, achieving a processing throughput of millions of data entries per second. However, its existing data masking capabilities have not been specifically optimized for V2X scenarios, necessitating the development of a data masking solution for V2X environments to achieve both data security and efficiency. Summary of the Invention

[0005] This application provides a method, device, and medium for desensitizing V2X data in a vehicle-to-everything (V2X) context, to address the following technical problem: how to identify and desensitize sensitive data in a V2X scenario based on SeaTunnel.

[0006] In a first aspect, embodiments of this application provide a method for desensitizing V2X data in the Internet of Vehicles (IoV), characterized by comprising: acquiring multi-source data in a V2X scenario using the SeaTunnel native connector; determining the sensitivity level of each field in the multi-source data based on a predefined rule base and a machine learning classification model; and using a desensitization plugin to perform hierarchical desensitization of the fields based on their sensitivity level and business scenario to obtain desensitized data corresponding to the fields.

[0007] Secondly, embodiments of this application also provide a vehicle-to-everything (V2X) data desensitization device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a vehicle-to-everything (V2X) data desensitization method as described in the first aspect above.

[0008] Thirdly, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement a V2X data desensitization method as described in the first aspect above.

[0009] The V2X data desensitization method, device, and medium provided in this application have the following beneficial effects: In this embodiment, multi-source data in a V2X scenario can be obtained through the SeaTunnel native connector. Then, based on a predefined rule base and a machine learning classification model, the sensitivity level of each field in the multi-source data is determined. Finally, a de-identification plugin is used to perform graded de-identification of the fields based on their sensitivity level and business scenario, obtaining the de-identified data corresponding to the aforementioned fields. In this way, based on SeaTunnel, the identification and de-identification of sensitive data in a V2X scenario can be achieved. Through this method, both data security and de-identification efficiency for V2X sensitive data can be ensured simultaneously. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a V2X data anonymization method provided in this application embodiment; Figure 2 An architecture diagram of a V2X data desensitization system provided in this application embodiment; Figure 3 This is a schematic diagram of the internal structure of a V2X data desensitization device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] According to the mandatory provisions of GB 44495-2024 "Data Security Requirements for Internet of Vehicles Information Services", V2X data must be irreversibly anonymized, and the transmission latency must be strictly controlled within 100 milliseconds. At the same time, the "Data Security Law of the People's Republic of China" and the "Personal Information Protection Law of the People's Republic of China" also put forward clear compliance requirements for the whole-link audit and privacy protection of sensitive data.

[0013] In existing technologies, the compliance coverage of data anonymization is not comprehensive. It only meets the requirements of national standards, does not fully adapt to the audit traceability provisions in the Data Security Law and the Personal Information Protection Law, and lacks the adaptability for private cloud deployment.

[0014] This application provides a method for desensitizing V2X data in vehicle networking. The technical solution proposed in this application will be described in detail below with reference to the accompanying drawings.

[0015] Figure 1 This is a flowchart illustrating a V2X data anonymization method provided in an embodiment of this application. Figure 1 As shown in the figure, the V2X data anonymization method provided in this application embodiment specifically includes the following steps: Step 101: Use the SeaTunnel native connector to acquire multi-source data in the V2X scenario.

[0016] In practical applications, the V2X data anonymization method provided in this application can be applied to a V2X data anonymization system or to a modified SeaTunnel, with no specific limitations. In the embodiments of this application, the SeaTunnel native connector can be used to acquire multi-source data in V2X scenarios. That is, by utilizing the native connector provided by the SeaTunnel open-source data integration platform, various heterogeneous data sources involved in V2X technology can be directly connected and synchronized, facilitating the acquisition of data covering all scenarios such as vehicles, road infrastructure, and cloud networks. The SeaTunnel native connector is a standardized data interface provided by SeaTunnel, supporting reading data from various data sources (such as relational databases, NoSQL, file systems, message queues, etc.) or writing data to the target system. Thus, a single toolkit using the SeaTunnel native connector can solve the problem of acquiring data across all V2X scenarios, avoiding the complexity of splicing multiple toolchains. Furthermore, it supports millisecond-level data synchronization, meeting real-time requirements and achieving low latency. Moreover, it has strong parallel processing capabilities, capable of handling explosive growth in massive amounts of device data and achieving high throughput.

[0017] Step 102: Based on a predefined rule base and a machine learning classification model, determine the sensitivity level of each field in the multi-source data.

[0018] In this embodiment, the sensitivity level of each field in the multi-source data can be determined using a predefined rule base and a machine learning classification model, rather than simply relying on manually defined rules. This improves the ability to identify sensitive data in multi-source data.

[0019] Step 103: Using a desensitization plugin, based on the sensitivity level of the field and the business scenario, perform hierarchical desensitization on the field to obtain the desensitized data corresponding to the field.

[0020] In this embodiment, a de-identification plugin can be used to perform tiered de-identification of the aforementioned fields based on their sensitivity levels and business scenarios, thereby obtaining the corresponding de-identified data. This embodiment uses a plugin instead of directly using algorithms for de-identification because direct algorithmic de-identification is suitable for simple rules that do not require dynamic adjustment, but it has high maintenance costs and poor scalability. Furthermore, sensitive parameters such as keys and salt values ​​involved in the de-identification process may be hard-coded in the code, posing a high risk of leakage. Rule changes also require code permissions, potentially leading to abuse by internal personnel, resulting in low overall security. However, using a de-identification plugin is not only suitable for complex, dynamic, and compliance-critical scenarios like V2X, but also allows for flexible rule management and security control through centralized configuration, resulting in high security. Furthermore, tiered de-identification based on the sensitivity levels and business scenarios allows for differentiated de-identification processing according to the sensitivity levels of the fields, maintaining the data's value to the business while protecting privacy, achieving a balance between privacy and utility, and ensuring that the de-identified multi-source data (de-identified data) can support the corresponding business.

[0021] In this embodiment, multi-source data in a V2X scenario can be obtained through the SeaTunnel native connector. Then, based on a predefined rule base and a machine learning classification model, the sensitivity level of each field in the multi-source data is determined. Finally, a de-identification plugin is used to perform graded de-identification of the fields based on their sensitivity level and business scenario, obtaining the de-identified data corresponding to the aforementioned fields. In this way, based on SeaTunnel, the identification and de-identification of sensitive data in a V2X scenario can be achieved. Through this method, both data security and de-identification efficiency for V2X sensitive data can be ensured simultaneously.

[0022] In one possible implementation, the acquisition of multi-source data in a V2X scenario using the SeaTunnel native connector may include: Receive real-time sensing data from roadside devices and vehicle sensors via connector-kafka or connector-mqtt; Obtain structured data from the vehicle terminal based on connector-cdc-mysql; Use connector-file-local to obtain semi-structured image metadata from the dashcam; The configuration command data of the cloud V2X service platform is obtained through connector-http.

[0023] In practical applications, V2X data is information generated and shared when vehicles interact with their surrounding environment (including other vehicles, pedestrians, infrastructure, networks, etc.) in real time. The following are the main types of V2X data and their specific contents: (1) Vehicle-to-Vehicle (V2V) data, including dynamic information such as vehicle speed, position, direction, acceleration, and braking status. (2) Vehicle-to-Infrastructure (V2I) data, specifically including traffic signal information (including traffic light status, countdown, right-of-way, etc.), road condition information (such as slippery road surface, potholes, construction areas, etc.), parking and charging information (such as parking space location, charging pile status, etc.). Traffic management information: such as speed limits, congestion warnings, route suggestions, etc. These data can help vehicles choose the optimal driving route and alleviate traffic congestion. (3) Vehicle-to-Person (V2P) data, including pedestrian location and movement trajectory, etc. (4) Vehicle-to-Network (V2N) data, including real-time traffic information, remote diagnostics and software updates, etc.

[0024] In practical applications, the SeaTunnel native connector can be used to access V2X data sources across all scenarios, acquiring multi-source data to cover real-time protocols and structured storage. Specifically, it can receive real-time perception data (such as obstacle information and vehicle dynamic status) from roadside units (RSUs) and onboard sensors via connector-kafka / connector-mqtt; synchronize structured data from onboard terminals (such as Vehicle Identification Numbers (VINs), owner's mobile phone number, and license plate number) via connector-cdc-mysql; read semi-structured image metadata from dashcams using connector-file-local; and access configuration command data from the cloud-based V2X service platform via connector-http. An RSU is a type of urban transportation infrastructure that provides internet connectivity and traffic information services through roadside units. The VIN, also known as the vehicle identification number or 17-digit code, is a unique identification code assigned to each vehicle by the manufacturer. Acquiring this multi-source data facilitates subsequent data anonymization and ensures data security.

[0025] For example, the configuration example (MQTT access) is as follows: source: MqttSource: broker: tcp: / / vehicle-sensor-1:1883 topic: v2x_vehicle_dynamic client.id: seaTunnel-v2x-001 schema: fields: license_plate: string gps_coordinate: string vin: string speed: int timestamp: long In one possible implementation, determining the sensitivity level of each field in the multi-source data based on a predefined rule base and a machine learning classification model includes: The field names of each field in the multi-source data are matched with the sensitive field dictionary in the predefined rule base to obtain the data type and confidence level of the field. If the confidence level is greater than a preset threshold, the sensitivity level of the field is determined based on the data type of the field. If the confidence level is less than or equal to the preset threshold, a machine learning classification model is used to determine the sensitivity level of the field based on the data format and data purpose of the field.

[0026] In practical applications, a dual-drive identification mechanism combining a rule base and machine learning can be constructed to automatically classify and label sensitive data. The predefined rule base can include a dictionary of sensitive V2X fields (such as "vin", "license_plate", "gps_coordinate", "owner_id"). This allows for preliminary identification of the data type (data category) of a field based on the field name matching across multiple data sources. It can also determine the confidence level. For example, when a field name completely matches a sensitive field in the predefined rule base (such as "vin" or "license_plate"), the confidence level can be directly set to 1.0 (highest certainty). In this case, the sensitivity level of the field can be directly determined based on its data type. If the field name partially matches (e.g., "vehicle_vin" contains "vin") but does not meet the exact matching standard, the confidence level may be set to 0.8-0.9, requiring further evaluation. In practical applications, the preset threshold can be determined based on the actual situation; for example, it could be 0.9. When the confidence level is less than or equal to the preset threshold, a machine learning classification model can be used. For example, integrating the LightGBM classifier can automatically label the sensitivity level (level 1-5, with level 5 being the highest) based on the data format (such as a 17-digit VIN code or an 11-digit mobile phone number) and data purpose (such as "for security alerts" or "for traffic statistics"). This ensures that the corresponding sensitivity level is determined for each field.

[0027] In practical applications, a dynamic update mechanism can be implemented, that is, the training samples of the above-mentioned predefined rule base and model are updated regularly (e.g., quarterly) based on newly added V2X data fields (such as the battery status of new energy vehicles) to ensure that the recognition accuracy is ≥99.5%. For example, the core logic code for identifying multi-source data is as follows: public class V2XSensitiveRecognizer { private RuleBasedDetector ruleDetector; private LightGBMClassifier mlModel; public SensitiveLabel recognize(SeaTunnelRow row) { / / Preliminary identification by the rule base SensitiveLabel ruleLabel = ruleDetector.detect(row); if (ruleLabel.getConfidence()>0.9) { return ruleLabel; } / / Machine Learning Secondary Calibration FeatureVector features = extractFeatures(row); / / Extract field length, format, and usage tags return mlModel.predict(features); } } In one possible implementation, a desensitization plugin is used to perform hierarchical desensitization of the field based on its sensitivity level and business scenario, obtaining the desensitized data corresponding to the field, including: A desensitization plugin is used to determine whether the sensitivity level of the field is greater than or equal to the first preset level; If the sensitivity level of the field is greater than or equal to the first preset level, the field is desensitized using a desensitization algorithm according to the data type of the field to obtain the desensitized data corresponding to the field. The desensitization algorithm is determined based on the sensitivity level of the field and the business scenario.

[0028] In practical applications, a desensitization plugin can be developed, such as a V2X-specific desensitization Transform plugin (inheriting SingleFieldOutputTransform), which can then implement "hierarchical desensitization" based on sensitivity level and business scenario. The core strategy is shown in Table 1.

[0029] Table 1

[0030] The aforementioned data masking plugin is a modular component integrating multiple data masking algorithms. Through a unified interface and configuration-driven mechanism, it encapsulates different data masking strategies into pluggable units. The V2X-specific data masking Transform plugin is developed based on SingleFieldOutputTransform (the base class of Apache SeaTunnel) and belongs to the Transform node in the data pipeline. Its function is to perform data masking operations on specified fields during data flow, rather than being a standalone function.

[0031] In practical applications, different data types have different sensitivity levels and corresponding rules. Using compliant data masking algorithms can help enterprises automatically comply with these regulations and avoid legal risks and fines due to data breaches. In practice, a data masking plugin can be used to determine if a field's sensitivity level is greater than or equal to a first preset level. If the sensitivity level is greater than or equal to the first preset level, the corresponding data masking algorithm is applied to the field based on its data type to obtain the masked data. By setting a first preset level, it ensures that all sensitive fields reaching or exceeding that level are masked, effectively preventing the leakage of sensitive information. Furthermore, the data masking plugin can preserve the original format and structure of the data as much as possible during the masking process, ensuring that the masked data can still be correctly identified and processed by business systems. This helps maintain business continuity and data availability, avoiding data unavailability or business interruption due to data masking. Moreover, in practical applications, the data masking plugin can automate the masking process based on configured algorithms, reducing the need for manual intervention and operation. This lowers the error rate and risk during the masking process while improving efficiency.

[0032] In one possible implementation, the de-identification plugin reads a YAML configuration file upon startup to construct an internal data structure and obtain the de-identification algorithm and parameters corresponding to each field. The YAML configuration file allows adjustment of the de-identification algorithm based on the data source and business scenario of the field.

[0033] In practical applications, the data masking strategy (data masking algorithm) is configurable. Field-level data masking rules can be defined through YAML configuration files, supporting dynamic adjustments based on data source and business scenario, as shown in the following example: transform: - type: v2x_mask fields: - name: license_plate algorithm: partial_mask options: { "prefix_length": 2, "suffix_length": 1, "mask_char": "*"} - name: gps_coordinate algorithm: gaussian_blur options: { "blur_radius": 0.02, "admin_level": "district"} # Blur to the administrative district level - name: vin algorithm: fpe_encrypt options: { "key_version": "v202511"} sensitive_level_filter: 3 # Only process data with a sensitivity level ≥ 3 In practical applications, YAML (YAML Ain't Markup Language) is a human-readable data serialization format designed specifically for configuration files, featuring a concise and easy-to-read syntax and powerful expressive capabilities.

[0034] In practical applications, you can also ask users for their opinions. For example, you can display two or three algorithms (all compliant) for de-identifying a certain data type on the page, and then determine the de-identification algorithm based on the user's choice. This helps to improve user satisfaction.

[0035] In practical applications, the masking plugin works as follows: Upon startup, it reads the YAML configuration file, constructs an internal data structure (such as a Map<field name, masking rule>), and records the masking algorithm and parameters for each field. This masking plugin is embedded in the data pipeline (SeaTunnel) and processes each field of each data entry according to the following steps: (1) Field Filtering: Filter by sensitivity level. Check if the sensitivity level of a field meets the configured filtering conditions (e.g., sensitive_level_filter: 3 only processes fields with a sensitivity level ≥ 3). Example: If a field has a sensitivity level of 2, then skip the desensitization process.

[0036] (2) Policy routing: Matching de-identification algorithms. Based on the field name or data type, the corresponding de-identification algorithm and parameters are found in the configuration.

[0037] (3) Algorithm execution: Call the corresponding desensitization method, execute the specific logic according to the algorithm type, and dynamically pass in the configuration parameters.

[0038] (4) Result returned: Replace the original field. Write the de-identified value back to the data row and continue to the next pipeline node.

[0039] Thus, the aforementioned desensitization plugin can support a variety of desensitization algorithms and dynamic parameters, and can be seamlessly integrated with data pipelines to process data streams in batches. It can also ensure data security based on sensitivity level filtering and key management.

[0040] In practical applications, for example, the core code implementation of the above process is as follows: public class V2XMaskTransform extends SingleFieldOutputTransform { private final String dataType; @Override protected Object getOutputFieldValue(SeaTunnelRowAccessor inputRow) { String original = inputRow.getFieldAsString(sourceFieldIndex); switch (dataType) { case "POSITION": return blurPosition(original); / / Implement Gaussian blur algorithm case "VIN": return FPE.encrypt(original, getDynamicKey()); / / Dynamic key encryption default: return maskCommon(original); } } } In practical applications, YAML configuration defines field names, algorithm types, and parameters (such as mask length, blur radius, and key version) to decouple the strategy from the code. In the code above, the field type is determined by `dataType`, and the corresponding algorithm is called (e.g., `POSITION` triggers Gaussian blur, `VIN` triggers FPE encryption), while other types use a partial mask by default. FPE encryption supports dynamic keys and can be used with a key management system to implement key rotation. This avoids the problem of data unavailability caused by a "one-size-fits-all" approach.

[0041] In one possible implementation, after employing a desensitization plugin to perform hierarchical desensitization on the field based on its sensitivity level and business scenario, and obtaining the desensitized data corresponding to the field, the method further includes: The anonymized data is verified and audited to ensure that it meets multi-dimensional compliance requirements; Based on the intended use and deployment environment of the anonymized data, the anonymized data is distributed to the target application via the SeaTunnel connector.

[0042] In the above embodiments, after obtaining the anonymized data, it can be verified and audited to ensure that the anonymized data complies with these regulatory standards in all dimensions, avoiding legal risks and hefty fines due to data mishandling. Furthermore, depending on the specific purpose and deployment environment, the anonymized data can be distributed to target applications via the SeaTunnel connector. The SeaTunnel connector has a rich plugin ecosystem, capable of connecting various data sources and target applications (such as data warehouses, big data analytics platforms, and business systems). This allows anonymized data to be quickly and efficiently transmitted to different target locations according to specific needs, achieving data integration and sharing. Moreover, SeaTunnel provides a visual monitoring and management interface that can monitor the progress, status, and performance indicators of data distribution in real time. Administrators can intuitively understand the data distribution status through the interface, promptly identify and resolve problems, and ensure the stable operation of data distribution tasks.

[0043] In one possible implementation, the verification and auditing of the de-identified data includes: The anonymized data is subjected to format verification, correlation verification, and compliance verification. If the verification is successful, a processing log is generated and stored based on the information in the de-identified data; A compliance report is generated based on the processing results of the de-identified data through a third-party audit.

[0044] In practical applications, a three-dimensional compliance system of "real-time verification + full-link logging + periodic auditing" can be constructed to cover multi-dimensional compliance requirements. In the real-time verification unit, a rule engine can be integrated to perform triple verification: First, format verification (e.g., maintaining a 17-digit VIN code and retaining the prefix / suffix format after masking the license plate number). Format verification checks whether the anonymized data conforms to predetermined data format specifications, such as date, number, and string formats. This ensures that the anonymized data can be correctly identified and parsed during subsequent processing and analysis, avoiding data processing failures or erroneous results due to format errors, thereby improving data usability. Second, correlation verification (e.g., the inability to combine blurred GPS coordinates with vehicle speed and timestamps to deduce the vehicle's precise trajectory). Correlation verification mainly checks whether the logical relationships between different fields and records in the anonymized data are correct, ensuring data integrity. This helps avoid data loss or errors caused by the anonymization operation, ensuring that the data reflects the true situation and passes correlation verification, ensuring that the anonymized data can still support complex business analysis needs. Finally, there is compliance verification (such as ensuring the SM4 encryption algorithm complies with national cryptographic standards and that log fields meet the audit requirements of the Data Security Law). Compliance verification ensures that the anonymized data complies with these regulatory requirements during processing.

[0045] In practical applications, a full-link traceability unit can be constructed. Under the condition that the above verification is passed, the processing logs are stored through connector-elasticsearch. Each processing log includes: (1) basic information: processing time, data source (such as "vehicle sensor-001"), desensitization algorithm type, key version; (2) data sample: original data (sampling rate of 1% before desensitization to avoid storing the full original data), data after desensitization, sensitivity level; (3) operation record: configuration modifier, rule effective time, and abnormal handling result. The processing log records the processing process of desensitized data in detail. When problems occur or investigations are needed, the source of the problem can be quickly located by viewing the processing log, and the details of data processing can be understood to achieve traceability of operations. Moreover, the processing log provides an important basis for internal auditing and supervision of enterprises. Auditors can analyze the processing logs to assess the compliance and security of data processing, discover potential risks and problems, and take timely measures to rectify them to ensure that the enterprise's data management work complies with internal regulations and external regulatory requirements. When problems arise with de-identified data, processing logs can help technicians quickly pinpoint the source and cause of the problem, enabling targeted troubleshooting and repair.

[0046] In practical applications, third-party audit access capabilities can be provided to automatically generate compliance reports periodically (e.g., quarterly), covering: (1) de-identification coverage rate (≥99.9%) and abnormal data processing rate (100%); (2) key update records and log integrity verification results; (3) compliance with GB 44495-2024 and the Data Security Law. In practical applications, third-party audit institutions are independent and professional, and can objectively evaluate the processing results of de-identified data. The compliance reports generated by third-party audits can serve as important proof of enterprise data compliance, demonstrating that the enterprise complies with relevant laws and standards in data processing, and enhancing the enterprise's compliance evidentiary power.

[0047] In one possible implementation, distributing the anonymized data to the target application via the SeaTunnel connector, based on the data purpose and deployment environment of the anonymized data, includes: The anonymized statistical analysis data is stored in the data lake via connector-hudi for offline analysis. The anonymized real-time business data is pushed to the traffic management platform and vehicle terminals via connector-redis; The anonymized compliance filing data is written to the government database via connector-jdbc to meet regulatory filing requirements; For the de-identified data with a sensitivity level greater than or equal to the second preset level, a connector-console alarm is triggered and pushed to the security administrator via connector-email; The de-identification plugin and the processing log storage are deployed on an enterprise private cloud cluster to avoid data transmission across the public network.

[0048] In practical applications, SeaTunnel connectors can distribute data to target storage / applications based on data usage and deployment environment, supporting both private and hybrid cloud deployments. Anonymized statistical analysis data (such as traffic flow and regional vehicle speed distribution) can be stored in a data lake via connector-hudi for offline analysis; real-time business data (such as vehicle safety warning information) can be pushed to traffic management platforms and in-vehicle terminals via connector-redis; compliance and filing data (such as anonymized vehicle owner information) can be written to government databases via connector-jdbc to meet regulatory filing requirements; data with a sensitivity level greater than or equal to the second preset level (e.g., sensitivity level ≥ 5) can trigger connector-console alarms and be pushed to the security administrator via connector-email. In practical applications, de-identified GPS data (blurred down to the administrative region level) combined with vehicle speed data can be used for real-time traffic analysis and congestion prediction, avoiding the leakage of precise vehicle locations. De-identified VIN codes and license plate numbers (retaining some formatting) are used for vehicle identification in V2V communication, and combined with the blurred location to achieve collision warnings while preventing the leakage of identity information. When government departments (such as transportation bureaus and data bureaus) exchange de-identified data across systems, compliance can be proven through periodic audit reports without transmitting original sensitive information. De-identified vehicle dynamic data (such as speed and battery status) is used for vehicle performance optimization analysis, supporting R&D needs while protecting vehicle owner privacy.

[0049] In practical applications, the aforementioned anonymized data can also be adapted for private cloud deployment, supporting the deployment of anonymization plugins and processing log storage on enterprise private cloud clusters, avoiding data transmission across the public network. In practical applications, the aforementioned anonymized data supports public cloud, private cloud, and hybrid cloud deployments, adapting to the different data storage compliance requirements of connected vehicle companies (such as automakers and traffic management departments), reducing resource utilization by 30% compared to traditional solutions.

[0050] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a V2X data anonymization system, the structure of which is as follows: Figure 2 As shown, the system includes: a multi-source data access layer, a sensitive data dynamic identification layer, a dynamic de-identification processing layer, a compliance verification and traceability layer, and a multi-scenario output adaptation layer.

[0051] In practical applications, V2X data sources (RSU / vehicle / cloud) transmit multi-source heterogeneous data to a multi-source data access layer via Kafka / MQTT / CDC. This data is then transmitted to a sensitive data dynamic identification layer (rule base + machine learning) to label the multi-source data with sensitivity levels. Finally, it is transmitted to a dynamic de-identification layer (V2X-specific Transform). If successful, the de-identified data is input to a compliance verification and traceability layer (real-time verification + ES logs), and then to a multi-scenario output adaptation layer (Hudi / Redis / JDBC). At this point, the data can be distributed for business applications (traffic analysis / security alerts / compliance filing) or stored in private / public clouds. If the above fails, an alarm module (Console / Email) is activated. Kafka is a distributed streaming media platform typically used to build real-time streaming data pipelines that reliably acquire data between systems or applications, and to build real-time streaming applications that transform or respond to data streams. MQTT refers to Message Queuing Telemetry Transport, a lightweight message transmission protocol based on a publish / subscribe model. In the field of data processing, CDC usually refers to Change Data Capture, a technology used to capture and transmit data changes in a database.

[0052] This enables real-time access and dynamic de-identification of V2X multi-source data, meeting the flexibility, scalability, and efficiency requirements of vehicle network big data processing, while also taking into account data privacy protection and compliance requirements. It is suitable for scenarios such as intelligent traffic management, vehicle safety early warning, and compliant data sharing.

[0053] To make the technical solution of this invention clearer, the desensitization process is described in detail below with reference to specific testing cases. This embodiment takes the desensitization process in a vehicle safety warning scenario as an example.

[0054] a) Cluster architecture: 3-node SeaTunnel cluster (2 cores 8G / node, supports horizontal scaling to 10 nodes), with Zeta engine enabled (streaming mode). b) Dependencies: Enable the connector-kafka / connector-mqtt / connector-hudi / connector-elasticsearch and transform-v2x_mask plugins in plugin_config, and integrate the LightGBM model inference library (version 3.3.5). c) Private cloud deployment configuration: Close the public network access interface, implement cluster management through VPN, store the keys in the local encryption machine (compliant with the national cryptographic SM4 storage standard), and only retain the log data in the private cloud Elasticsearch cluster. Example of the desensitization process (vehicle safety warning scenario) 1) Vehicle A sends real-time data through V2I: license_plate=Su D40000, gps_coordinate=116.481028,39.912957, vin=LFV2A24G3J3001234, speed=60, timestamp=1730712000000; 2) The access layer receives the data through MQTT Source and parses it into the SeaTunnel Row format; 3) The identification layer marks the sensitivity levels: license_plate (level 4), gps_coordinate (level 4), vin (level 5), speed (level 2); 4) The desensitization layer executes the policies: 1. The license plate is desensitized to Su D4***0 (2 prefix digits and 1 suffix digit are retained); 2. The GPS coordinates are blurred to Chaoyang District, Beijing (116.48±0.02, 39.91±0.02) (Gaussian blur + administrative region mapping); 3. The VIN code is encrypted through FPE to LFV2A24G3J****34 (retain the format, irreversible); 4. The vehicle speed is not desensitized (sensitivity level 2, utility first); e) Verification layer verification: Confirm that the VIN code is 17 digits long and the GPS cannot reverse the exact location, and the verification passes; f) The output layer pushes the desensitized data to the traffic management platform through Redis Sink for vehicle safety warning; g) The traceability unit records the logs to Elasticsearch, including the original / desensitized data samples, algorithm types (partial_mask / fpe_encrypt), and key versions (v202511).

[0055] The above is the method embodiment and system proposed in this application. Based on the same inventive concept, the embodiments of this application also provide a vehicle networking V2X data desensitization device, the structure of which is as Figure 3 shown.

[0056] Figure 3This is a schematic diagram of the internal structure of a V2X data desensitization device provided in an embodiment of this application. Figure 3 As shown, the device includes: At least one processor 301; And a memory 302 that is communicatively connected to at least one processor; The memory 302 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 301 so that at least one processor 301 can: execute the above-described V2X data desensitization method.

[0057] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium stores computer-executable instructions, which are configured to execute the above-mentioned V2X data desensitization method.

[0058] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0059] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0065] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0066] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0067] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0068] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for desensitizing V2X data in vehicle-to-everything (V2X) communication, characterized in that, include: Use the SeaTunnel native connector to acquire multi-source data in V2X scenarios; Based on a predefined rule base and a machine learning classification model, the sensitivity level of each field in the multi-source data is determined. Using a desensitization plugin, the fields are desensitized in a tiered manner based on their sensitivity level and business scenario to obtain the desensitized data corresponding to each field.

2. The method according to claim 1, characterized in that, The acquisition of multi-source data in V2X scenarios using the SeaTunnel native connector includes: Receive real-time sensing data from roadside devices and vehicle sensors via connector-kafka or connector-mqtt; Obtain structured data from the vehicle terminal based on connector-cdc-mysql; Use connector-file-local to obtain semi-structured image metadata from the dashcam; The configuration command data of the cloud V2X service platform is obtained through connector-http.

3. The method according to claim 1, characterized in that, The determination of the sensitivity level of each field in the multi-source data based on a predefined rule base and a machine learning classification model includes: The field names of each field in the multi-source data are matched with the sensitive field dictionary in the predefined rule base to obtain the data type and confidence level of the field. If the confidence level is greater than a preset threshold, the sensitivity level of the field is determined based on the data type of the field. If the confidence level is less than or equal to the preset threshold, a machine learning classification model is used to determine the sensitivity level of the field based on the data format and data purpose of the field.

4. The method according to claim 1, characterized in that, The method employs a data masking plugin to perform tiered data masking on the fields based on their sensitivity level and business scenario, obtaining the masked data corresponding to each field, including: A desensitization plugin is used to determine whether the sensitivity level of the field is greater than or equal to the first preset level; If the sensitivity level of the field is greater than or equal to the first preset level, the field is desensitized using a desensitization algorithm according to the data type of the field to obtain the desensitized data corresponding to the field. The desensitization algorithm is determined based on the sensitivity level of the field and the business scenario.

5. The method according to claim 4, characterized in that, When the de-identification plugin starts, it reads the YAML configuration file to build an internal data structure and obtains the de-identification algorithm and parameters corresponding to each field. The YAML configuration file supports adjusting the de-identification algorithm according to the data source and business scenario of the field.

6. The method according to claim 1, characterized in that, After employing a de-identification plugin to perform tiered de-identification of the field based on its sensitivity level and business scenario, and obtaining the de-identified data corresponding to the field, the method further includes: The anonymized data is verified and audited to ensure that it meets multi-dimensional compliance requirements; Based on the intended use and deployment environment of the anonymized data, the anonymized data is distributed to the target application via the SeaTunnel connector.

7. The method according to claim 1, characterized in that, The verification and auditing of the de-identified data includes: The anonymized data is subjected to format verification, correlation verification, and compliance verification. If the verification is successful, a processing log is generated and stored based on the information in the de-identified data; A compliance report is generated based on the processing results of the de-identified data through a third-party audit.

8. The method according to claim 7, characterized in that, The step of distributing the anonymized data to the target application via the SeaTunnel connector, based on the data purpose and deployment environment of the anonymized data, includes: The anonymized statistical analysis data is stored in the data lake via connector-hudi for offline analysis. The anonymized real-time business data is pushed to the traffic management platform and vehicle terminals via connector-redis; The anonymized compliance filing data is written to the government database via connector-jdbc to meet regulatory filing requirements; For the de-identified data with a sensitivity level greater than or equal to the second preset level, a connector-console alarm is triggered and pushed to the security administrator via connector-email; The de-identification plugin and the processing log storage are deployed on an enterprise private cloud cluster to avoid data transmission across the public network.

9. A V2X data anonymization device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform a V2X data desensitization method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, a V2X data desensitization method as described in any one of claims 1-8 is implemented.

Images