Data processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and a program product. In the embodiment of the invention, after business data uploaded by target data acquisition equipment is received, a target DOM tree corresponding to target equipment is determined from a plurality of pre-constructed DOM trees; generating a path selector according to a to-be-converted field name in the service data, accurately positioning a target node in the target DOM tree by using the path selector, and obtaining a field value corresponding to the to-be-converted field name; and dynamically converting the field value corresponding to the to-be-converted field name into the field value under the field name used by the Internet of Things platform based on the conversion relationship corresponding to the target data acquisition device pre-maintained by the Internet of Things platform, so that the Internet of Things platform performs subsequent data processing, automatic conversion of data formats of different data acquisition devices is realized, and the data format conversion efficiency is improved. And the flexibility and expandability of the Internet of Things platform are improved, so that the response speed of the Internet of Things platform to service requirements in diversified application scenes is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, storage medium, and program product. Background Technology

[0002] The Internet of Things (IoT) platform, as the core hub for realizing the interconnection of everything, serves as a bridge between data acquisition devices and upper-layer application systems, used to receive and manage various types of data. With the continuous expansion of IoT application scenarios, it is necessary to access data collected by data acquisition devices from different manufacturers using various data formats. Among these, the data acquisition devices differ significantly in their data format definitions. This diversity necessitates that the IoT platform develop separate parsing logic for each message format during the integration process, greatly increasing the complexity of IoT platform development and reducing integration efficiency.

[0003] To address these issues, existing IoT platforms typically employ a standardized access specification approach, mandating that all data acquisition devices transmit data according to the platform's preset data format to ensure compliance with platform requirements. However, this standardized approach, by enforcing a uniform format, fails to adapt to the diverse needs of different industry scenarios, resulting in low flexibility and insufficient scalability of the IoT platform. Consequently, it reduces the platform's responsiveness to diverse application scenarios and business demands. Summary of the Invention

[0004] The embodiments of this application provide a data processing method, device, storage medium, and program product to achieve automatic conversion of data formats from different data acquisition devices, improve the flexibility and scalability of the Internet of Things (IoT) platform, and thereby improve the response speed of the IoT platform to business needs in diverse application scenarios.

[0005] This application provides a data processing method, device, storage medium, and program product applied to an Internet of Things (IoT) platform. The IoT platform pre-maintains the conversion relationship between field names used by the IoT platform and field names used by various data acquisition devices, and pre-constructs multiple DOM trees corresponding to the various data acquisition devices. Multiple nodes in the DOM trees store the field names and field values ​​used by the corresponding data acquisition devices. The method includes: receiving business data uploaded by a target data acquisition device, the business data including field names used by the target data acquisition device and to be converted; determining a target DOM tree corresponding to the target data acquisition device from the multiple DOM trees; generating a path selector based on the field names to be converted, and using the path selector to determine a target node from the target DOM tree, the target node being the node containing the field names to be converted; obtaining the field values ​​corresponding to the field names to be converted from the target node; and converting the field values ​​corresponding to the field names to be converted into field values ​​under the field names used by the IoT platform based on the conversion relationship corresponding to the target data acquisition device, so that the IoT platform can perform data processing based on the converted field names and field values.

[0006] This application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is able to implement the various steps in the data processing method provided in this application.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the various steps of the data processing method provided in this application.

[0008] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enable the processor to implement the various steps in the data processing method provided in this application.

[0009] In this embodiment, after receiving business data uploaded by the target data acquisition device, the target DOM tree corresponding to the target device is determined from multiple pre-built DOM trees; a path selector is generated based on the field names to be converted in the business data, and the path selector is used to accurately locate the target node in the target DOM tree and obtain the field value corresponding to the field name to be converted; based on the conversion relationship corresponding to the target data acquisition device pre-maintained by the IoT platform, the field value corresponding to the field name to be converted is dynamically converted into the field value under the field name used by the IoT platform for subsequent data processing by the IoT platform. This realizes the automatic conversion of data formats of different data acquisition devices, improves the flexibility and scalability of the IoT platform, and thus improves the response speed of the IoT platform to business needs in diverse application scenarios. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1a A flowchart illustrating a data processing method provided in an exemplary embodiment of this application; Figure 1b A schematic diagram of the architecture of a data processing method provided in an exemplary embodiment of this application; Figure 2 A schematic diagram illustrating the process of constructing a DOM tree, provided as an exemplary embodiment of this application; Figure 3 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0013] In existing technologies, IoT platforms typically employ a mandatory unified access standard, requiring all data acquisition devices to transmit data according to a preset data format. However, different scenarios may require data formats with varying complexity. A unified access standard cannot meet the diverse needs of different data formats, resulting in an inability for IoT platforms to flexibly adapt to the differentiated requirements of various industry scenarios. This leads to insufficient scalability and low data transmission efficiency in specific scenarios, thereby reducing the platform's responsiveness to diverse business demands. Furthermore, when new data acquisition devices acquire data in new formats, IoT platforms often require large-scale modifications and upgrades, which not only compromises scalability but also increases maintenance costs and technical risks.

[0014] To address the aforementioned technical issues, in this embodiment, after receiving business data uploaded by the target data acquisition device, a target DOM tree corresponding to the target device is determined from multiple pre-built DOM trees. A path selector is generated based on the field names to be converted in the business data. The path selector is used to accurately locate the target node in the target DOM tree and obtain the field value corresponding to the field name to be converted. Based on the conversion relationship corresponding to the target data acquisition device pre-maintained by the IoT platform, the field value corresponding to the field name to be converted is dynamically converted into the field value under the field name used by the IoT platform for subsequent data processing. This achieves automatic conversion of data formats from different data acquisition devices, improves the flexibility and scalability of the IoT platform, and thus improves the response speed of the IoT platform to business needs in diverse application scenarios.

[0015] Furthermore, when new data acquisition devices acquire data in new formats, the IoT platform does not need to undergo large-scale transformation and upgrades. It only needs to pre-maintain the conversion relationships corresponding to the new data acquisition devices, which not only improves scalability but also reduces maintenance costs and technical risks.

[0016] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0017] Figure 1a This is a schematic flowchart illustrating a data processing method provided in an exemplary embodiment of this application. The data processing method can be based on... Figure 1b The schematic diagram shown below illustrates the implementation of the data processing method provided in an exemplary embodiment of this application, as follows: Figure 1b As shown, the architecture includes: a target data acquisition device and an IoT platform; the IoT platform includes a path selector and a primary language model. (As shown...) Figure 1aAs shown, this data processing method is applied to an IoT platform. The IoT platform pre-maintains the conversion relationship between field names used by the IoT platform and field names used by various data acquisition devices, and pre-constructs multiple DOM (Document Object Model Tree) trees corresponding to various data acquisition devices. Multiple nodes in the DOM tree store the field names and field values ​​used by the corresponding data acquisition devices. The method includes: S101. Receive service data uploaded by the target data acquisition device. The service data includes the field names used by the target data acquisition device and to be converted. S102. From multiple DOM trees, determine the target DOM tree corresponding to the target data acquisition device; S103. Generate a path selector based on the field name to be converted, and use the path selector to determine the target node from the target DOM tree. The target node is the node where the field name to be converted is located. S104. Obtain the field value corresponding to the field name to be converted from the target node; S105. Based on the conversion relationship corresponding to the target data acquisition device, the field value corresponding to the field name to be converted is converted into the field value under the field name used by the IoT platform, so that the IoT platform can perform data processing based on the converted field name and field value.

[0018] The method provided in the embodiments of this application is applied to Figure 1a The IoT platform shown is a core data hub system designed for the Internet of Things (IoT) scenario. It acts as a bridge between data acquisition devices and upper-layer application systems, receiving, managing, and processing business data from various data acquisition devices, and providing standardized data services to upper-layer application systems.

[0019] Among them, the target data acquisition device is a terminal device or edge node that uploads raw business data to the IoT platform. It can acquire various business data such as temperature, humidity, and voltage, and transmit this business data to the IoT platform through network channels. The target data acquisition device is the most front-end data source in the IoT system, and the format and standardization of its data directly affect the overall operating efficiency of the IoT platform and the upper-layer application systems.

[0020] In this application embodiment, the specific type of the target data acquisition device is not limited. For example, the target data acquisition device can be a smart sensor deployed in an industrial site (such as a temperature and humidity transmitter with the Modbus RTU protocol), a smart meter, water meter, or gas meter in urban infrastructure, or a smart hardware device with data acquisition and uploading capabilities such as a vehicle terminal, a drone telemetry module, an agricultural environmental monitoring station, or a smart home gateway.

[0021] In this embodiment of the application, the specific format, structure, and content type of the business data are not limited. For example, the business data may be in JSON (JavaScript Object Notation) format, such as {"temp": 25.6, "hum": 65}; or it may be a hierarchical structured document based on the XML (eXtensible Markup Language) standard, such as <Data><Temperature>25.6< / Temperature>< / Data>.

[0022] It is worth noting that different data acquisition devices support significantly different data formats when transmitting business data. Business data is typically collected directly from the physical world in its initial state, and its format and structure vary depending on the manufacturer and model of the target data acquisition device, as well as its meaning depends on the specifications defined by the target data acquisition device. The differences in data formats are mainly reflected in the field definitions, data structures, and encoding methods within the business data.

[0023] For example, if the target data acquisition device is an industrial temperature and humidity sensor, the business data uploaded by this device might be {"t": 25.6, "hum": 65}, where the field names are "t" and "hum". However, if the target data acquisition device is the hardware of an agricultural meteorological station, the business data uploaded might be <Data>< / Data> <Temperature>25.6< / Temperature>< / Humidity>65< / Humidity>< / Data>, where the field names use full names and the structure is nested. Therefore, even for the same physical quantity (such as temperature), the field names (such as "temp", "temperature", or "T") and the specific numerical form (such as Celsius or Fahrenheit) may be completely different in different data acquisition devices.

[0024] Meanwhile, the specific formats of business data required by different IoT platform services vary depending on the application scenarios they serve. For example, in industrial automation scenarios, millisecond-level real-time requirements are needed, and the format of business data must be simple and efficient to support high-frequency status monitoring of the production line. In contrast, in smart healthcare scenarios, the integrity and security of business data formats become core requirements, necessitating complex formats with multiple checks to ensure the accuracy of patient vital signs information.

[0025] To address the significant differences in data formats supported by various data acquisition devices and the varying data format requirements of the application scenarios served by the IoT platform, this embodiment addresses this heterogeneity by pre-maintaining a conversion relationship between the standard field names used by the IoT platform and the native field names used by various data acquisition devices. This conversion relationship is a mapping rule established and stored in the IoT platform's configuration management. This rule explicitly specifies how to map the diverse field names used by data acquisition devices from different manufacturers and models to standard field names that the IoT platform can understand and use. Essentially, it creates a conversion relationship for each type of accessed data acquisition device, ensuring that the IoT platform can correctly understand the same data meaning expressed by different business data formats.

[0026] The IoT platform pre-constructs multiple DOM trees corresponding to various data acquisition devices. These trees are pre-generated and stored by the user based on the application scenarios served by the IoT platform and the data formats required by the types of data acquisition devices. They contain root nodes, child nodes, and hierarchical relationships between nodes. Furthermore, multiple nodes in the DOM tree store the field names and values ​​used by the corresponding data acquisition devices. The node's label or attribute name is the field name, representing the data item identifier defined on the device side; while the specific content contained in the node is the field value, representing the corresponding physical quantity value, state, or descriptive information. In this way, the DOM tree uniformly transforms heterogeneous data into a traversable tree model. This DOM tree is easy to process and traverse, laying the foundation for subsequent operations. For example, if the data acquisition device is an industrial temperature and humidity sensor, and the business data uploaded by this target data acquisition device is in JSON format: {"device": {"id": "sensor_001"}, "data": {"t": 25.6, "hum": 65}}, the IoT platform will construct a corresponding DOM tree for it. The root node of this tree may contain a node named "device", which in turn contains a leaf node named "id" with the field value "sensor_001". Simultaneously, the root node also contains a node named "data", which contains two leaf nodes: one storing the field name "t" and the field value 25.6, and the other storing the field name "hum" and the field value 65.

[0027] In this embodiment, business data uploaded by the target data acquisition device is received. This business data includes field names used by the target data acquisition device and to be converted. Specifically, the field names to be converted refer to the field names that need to be converted to be used by the IoT platform. More specifically, the field names to be converted are the actual field names on the target data acquisition device side corresponding to each standard field name used by the IoT platform in a pre-maintained conversion relationship.

[0028] For example, the IoT platform uses the field names "temperature" and "humidity," and the platform is configured with a conversion relationship between the target data acquisition device and the target data acquisition device: converting the field name "t" used by the target data acquisition device to "temperature," and converting the field name "hum" to "humidity." Therefore, when the target data acquisition device uploads business data in JSON (JavaScript Object Notation) format with the content: {"t": 25.6, "hum": 65}, the field names "t" and "hum" in this business data are the field names to be identified and processed, representing the specific manifestations of the IoT platform's field names "temperature" and "humidity" in the target data acquisition device's data.

[0029] The purpose of data processing is to accurately locate these field names to be converted, and based on the pre-maintained conversion relationship between the field names used by the IoT platform and the field names used by various data acquisition devices, to obtain the field values ​​corresponding to the field names to be converted in the target DOM tree, and to convert the field names to be converted and the field values ​​corresponding to the field names into the field names and field values ​​under the field names used by the IoT platform, thereby completing the data processing process.

[0030] The target DOM tree is determined from multiple pre-built DOM trees corresponding to various data acquisition devices, specifically the DOM tree corresponding to the target data acquisition device. In this embodiment, the specific determination method is not limited, as long as an accurate correspondence between the target data acquisition device and the DOM tree can be established. For example, the unique identifier of the target data acquisition device (such as deviceId or sn) contained in the received business data can be used to match and search within the multiple pre-built DOM trees corresponding to various data acquisition devices; alternatively, the business data can carry a field indicating the data format version used (such as formatVersion: "1.1"), and the IoT platform can load the corresponding DOM tree based on this version number.

[0031] Based on the target DOM tree described in the above embodiments, a path selector can be generated according to the field name to be converted. The path selector can be a dynamic query mechanism that incorporates semantic parsing capabilities. Using this dynamic query mechanism, target nodes can be determined from the target DOM tree. A target node is a specific node in the target DOM tree that stores a particular field name to be converted and its corresponding field value. The target node directly corresponds to the field name to be converted and is a container that carries the actual data content of that field.

[0032] By combining a path selector with the target DOM tree, the target node can be quickly located, avoiding the need for layer-by-layer traversal of the entire business data and complex conditional judgments required by traditional parsing methods, thus greatly improving data processing efficiency. At the same time, the path selector provides a precise positioning method, ensuring accurate extraction of the required data values ​​and reducing the possibility of data extraction errors.

[0033] Furthermore, after the path selector successfully locates the target node, the IoT platform extracts the specific data value directly associated with the field name to be converted from that target node; that is, it retrieves the field value corresponding to the field name to be converted. For example, the target DOM tree is constructed from JSON data {"sensor": {"temperature": 25.6}}; the target node located by the path selector stores the field name "temperature"; retrieving the field value means extracting the specific value 25.6 corresponding to the field name "temperature" from this target node.

[0034] In this embodiment, after successfully obtaining the field value corresponding to the field name to be converted from the target node, a standardized mapping is performed based on the conversion relationship corresponding to the target data acquisition device. The core of the process of converting the field value corresponding to the field name to be converted into the field value under the field name used by the IoT platform is not to change the field name and field value themselves, but to complete a "transfer of ownership" and "semantic alignment". That is, the field name and field value are handed over from the target data acquisition device to the IoT platform. After the transfer, the business data is regarded as understandable and processable data on the IoT platform.

[0035] Specifically, standardized mapping is the process of converting the field values ​​corresponding to the field names to be converted into field values ​​under the field names used by the IoT platform. This process includes dual conversion logic: on the one hand, the field names used by the target data acquisition device (such as "t") are mapped to the field names used by the IoT platform (such as "temperature_celsius"); on the other hand, the field values ​​corresponding to the field names to be converted are adapted to conform to the format standards that the IoT platform can use, ensuring that the converted business data can be stored, analyzed, and processed by the IoT platform.

[0036] It is worth noting that, in this embodiment, the specific conversion of the field value corresponding to the field name to be converted into the field value under the field name used by the IoT platform is not limited. This process is highly adaptable and can cover various data normalization needs. The following are two optional conversion embodiments: Optional Example 1: Time format processing.

[0037] Specifically, the business data uploaded by the target data acquisition device includes time, and the corresponding field value for the field name to be converted is a time string. These time strings come in a wide variety of formats (e.g., "2023-10-26 14:30:00", "20231026T143000Z"). The IoT platform can pre-set a set of common time formats and, through matching and parsing, convert various time string formats into a standard 13-digit Unix timestamp (e.g., 1698323400000) to ensure consistency in time processing within the IoT platform.

[0038] Optional Implementation Example 2: Processing of quality field values.

[0039] Specifically, for field names representing the status or data quality of a target data acquisition device, their corresponding field values ​​may differ from the standard semantics defined by the IoT platform. For example, the target data acquisition device might use 0 to represent "good," while the IoT platform standard defines it as "Good." In this case, the IoT platform pre-maintains the conversion relationships between the field names used by the IoT platform and those used by various data acquisition devices (e.g., 0 → "Good", 1 → "Bad") to ensure that business logic can make judgments based on the same type of status semantics.

[0040] Importantly, the method described in the above embodiments does not rely on the business data uploaded by the target data acquisition device. By establishing a conversion relationship between the field names used by the IoT platform and the field names used by the target data acquisition device, it can flexibly adapt to the format changes of business data uploaded by different data acquisition devices and different manufacturers. This eliminates the need to write dedicated parsing code for each business data format, thus reducing system development and maintenance costs. Furthermore, the IoT platform no longer requires each data acquisition device vendor to process all business data according to specified field names and values ​​before uploading, significantly improving access efficiency. Moreover, the converted data can be directly used by the IoT platform for other data processing, thereby improving data processing efficiency and the overall performance of the IoT platform.

[0041] Meanwhile, the combination of the DOM tree structure and path selector mechanism provides excellent scalability. Specifically, when processing business data from new data acquisition devices or adding new conversion relationships to the existing conversion relationships between field names used in the IoT platform and field names used by various data acquisition devices, it is only necessary to add the corresponding field name mapping relationship to the conversion relationship and adjust the path selector expression appropriately. No large-scale modifications to the core processing logic are required. This greatly facilitates system function expansion and version upgrades, while reducing the technical risks introduced by system modifications.

[0042] like Figure 1b As shown, the IoT platform includes a path selector, which can call the Large Language Model (LLM). The Large Language Model (LLM) is an advanced natural language processing model based on deep learning. It is usually pre-trained on massive amounts of text data and has powerful semantic understanding, contextual reasoning, and multimodal processing capabilities.

[0043] In one optional embodiment, the nodes on the DOM tree also include semantic tags corresponding to field names and field values; determining the target node from the target DOM tree using a path selector includes: calling the first language model to perform semantic parsing on the field name to be converted to obtain the semantic information to be queried; using the path selector, performing semantic traversal on the target DOM based on the semantic information to be queried and the semantic tags of each node in the target DOM tree to obtain a set of candidate nodes; constructing a subtree based on the hierarchical relationship between the nodes in the candidate node set; and using the path selector, traversing the subtree based on the field name to be converted to obtain the target node including the field name to be converted.

[0044] Semantic tags are additional attributes of nodes in the DOM tree, used to describe the actual semantic meaning of the node in the business scenario, rather than simply reflecting the original field name on the data acquisition device. For example, the field name "t" used by the data acquisition device is labeled with the semantic tag "temperature", and "hum" is labeled with "humidity". Similarly, the semantic tag for the field name "temp" might be "temperature", and the semantic tag for the field name "RH" might be "relative humidity". This tagging mechanism enables the IoT platform to understand the differentiated expressions of the same physical quantity by different data acquisition devices, forming the basis for intelligent matching. In the DOM tree, each node, in addition to storing the field name and field value, also contains one or more semantic tags. These semantic tags are automatically generated by a pre-trained second-largest language model, ensuring the accuracy and business relevance of the tags.

[0045] Calling the first major language model to perform semantic parsing on the field names to be converted, and obtaining the semantic information to be queried, refers to leveraging the powerful natural language understanding capabilities of the first major language model to parse potentially abbreviated or non-standard field names into one or more standardized semantic concepts with clear and general meanings. For example, the field name to be converted, "Hum", after being parsed by the first major language model, might yield the semantic information "humidity". As another example, the field name to be converted, "internal temperature", can be parsed into the semantic information "temperature", and the field name to be converted, "vol", might be parsed into "voltage".

[0046] Based on the semantic information to be queried obtained from the above embodiments and the semantic tags of each node in the target DOM tree, a path selector performs semantic traversal on the target DOM to obtain a candidate node set. Semantic traversal refers to the process of matching the semantic information to be queried obtained in the previous step with the pre-defined semantic tags on each node in the entire target DOM tree to find all semantically matching nodes. This process focuses on the similarity of semantic meaning, rather than the complete consistency of field name strings. The candidate node set is the set of nodes selected through semantic traversal that match all semantic tags with the semantic information to be queried. The hierarchical relationship between nodes in the candidate node set refers to their positions in the target DOM tree and their parent-child, sibling, and other structural relationships. For example, when the query semantic is "temperature", multiple nodes with semantic tags such as "temp", "temp_value", or "temperature" may be matched. These nodes may be scattered in the DOM tree, but they have hierarchical relationships (such as parent-child and sibling relationships). For example, in the target DOM tree, node A (semantic tag "temperature") and node B (semantic tag "temp_value") may be sibling nodes, both belonging to the parent node "sensor_data". This hierarchical relationship can reflect the structural logic of the business data and provide a basis for subsequent subtree construction.

[0047] Furthermore, based on the hierarchical relationships between nodes in the candidate node set obtained above, constructing a subtree is to narrow down the scope of subsequent secondary searches using the path selector, thereby improving efficiency. Since candidate nodes may be distributed across different branches of the target DOM tree, extracting them and their common parent nodes to form a new, smaller subtree avoids inefficient full-tree traversal within the massive target DOM tree. The specific logic for constructing the subtree can be referenced below for the logic of constructing a DOM tree based on the data type of field values ​​in the business data, and will not be elaborated upon here.

[0048] Furthermore, based on the subtree constructed above, a path selector traverses the subtree according to the field name to be converted to obtain the target node containing the field name to be converted. Before traversal, the path selector generates a precise path expression based on the semantic parsing result of the field name to be converted and its expected position in the subtree. This path expression adopts a hierarchical structure similar to a file path, which can uniquely identify the position of the target attribute in the tree.

[0049] For example, in the constructed subtree rooted at "device", the path selector generates the path expression "device.metrics.Humidity" based on the semantic information "humidity" and the field name "Humidity". This path expression guides the search from the subtree root node "device", through the intermediate node "metrics", and finally to the leaf node "Humidity". By executing this path expression, the node storing the field name "Humidity" can be quickly located, confirming it as the target node, and the corresponding field value can be retrieved. This precise traversal based on path expressions, performed within the narrowed subtree, ensures both search accuracy and significantly improves retrieval efficiency.

[0050] Figure 2 This is a schematic diagram illustrating the process of constructing a DOM tree, provided as an exemplary embodiment of this application. The process of constructing the DOM tree occurs before using the data processing method described in the above embodiments.

[0051] like Figure 2 As shown, the system receives business data uploaded by various data acquisition devices. The business data includes the field names and field values ​​used by the corresponding data acquisition devices. The system parses the field names and field values ​​in the business data based on a recursive traversal algorithm, and constructs a DOM tree based on the parsed field names and their corresponding field values, as well as the hierarchical relationships between the field names. The field names and field values ​​correspond to nodes in the DOM tree, and the hierarchical relationships between the field names correspond to the hierarchical relationships between nodes in the DOM tree.

[0052] The IoT platform continuously receives business data uploaded from various data acquisition devices. This business data contains raw information collected by each data acquisition device in specific scenarios, consisting of field names and corresponding field values ​​defined by the data acquisition devices. For example, a data acquisition device for a temperature sensor might upload {"temperature": 25.6, "unit": "℃"}, where "temperature" and "unit" are the field names used by the data acquisition device, and "25.6" and "℃" are the corresponding field values.

[0053] The IoT platform employs a recursive traversal algorithm for deep parsing of these structured business data. This recursive algorithm effectively handles complex data structures with nested hierarchies, i.e., the relationships between fields in the continuously received business data from the IoT, whether it's a multi-level JSON object or a complex format containing arrays. During parsing, it can identify all field names, field values, and their hierarchical relationships layer by layer in the business data, and construct the corresponding DOM tree based on this information. The hierarchical relationship between field names and field values ​​is the correspondence between the hierarchical organization logic defined by the field names and the field values. Specifically, the nesting or subordinate structure between field names (such as a parent field containing a child field) determines the hierarchical framework of the business data, and the field values ​​are embedded within this framework as the specific content of each field name. For example, in the business data JSON {"device": {"sensor": {"temp": 25.6}}}, the field name "temp" belongs to "sensor", and "sensor" belongs to "device", forming a hierarchical relationship.

[0054] The final constructed DOM tree has the following characteristics: each field name and field value is converted into an independent node in the DOM tree; the nested hierarchy between field names precisely corresponds to the hierarchical structure between nodes in the DOM tree; the root node of the DOM tree represents the entire business data; the child nodes correspond to the fields at each level; the leaf nodes store the specific field values; and the intermediate nodes reflect the organizational structure of the business data.

[0055] In one optional embodiment, a recursive traversal algorithm is used to parse the field names and values ​​in the business data, and a DOM tree is constructed based on the parsed field names, their corresponding field values, and the hierarchical relationships between the field names. This includes: based on the data type of the field values ​​in the business data, performing operations such as... Figure 2 The recursive traversal operation shown is as follows: S1. Create a virtual root node and use the virtual root node as the starting node for recursive traversal; S2. For the fields in the currently traversed business data, determine whether the data type of the current field is a basic type; S3. If the data type of the current field is a basic type, then create a leaf node and store the field name and field value in the leaf node; S4. If the data type of the field is an object type, then iterate through the key-value pairs contained in the field, create a corresponding child node for each key, and store the value corresponding to each key into the corresponding child node, and return to execute step S2. S5. If the data type of the field is an array, then iterate through each element in the array contained in the field, create a corresponding child node for each element, set an index attribute in the child node to mark the position of each element in the array, and store the value of each element in the corresponding child node, then return to execute step S2. S6. Set the virtual root node as the root node of the DOM tree, and add the child nodes generated by steps S3, S4 or S5 to the child node list of their respective parent nodes.

[0056] In step S1, the virtual root node serves as the abstract starting point for the entire data structure. The virtual root node itself does not contain actual fields from the business data; it is only used to uniformly manage the node hierarchy generated by subsequent parsing, ensuring a fixed traversal entry point regardless of the complexity of the business data structure. For example, when processing business data {"device": {"id": "S001", "sensors": [...]}}, the virtual root node "root" is first created as the starting point for recursive traversal. All subsequently parsed nodes (such as the "device" node) will be attached to this virtual root node, forming the top-level structure of the tree.

[0057] In step S2, the data type of the current field is checked to determine the subsequent processing path. Data types include basic types such as strings, numbers, and booleans, as well as composite types such as objects and arrays. The data type determination is based on the actual format of the business data. For example, in JSON data, "id":"S001" is a string, and "value":25.6 is a number. For instance, when iterating through the "device" field in the business data {"device":{"id":"S001","sensors":[...]}}, its type is determined to be an object type (not a basic type), thus triggering the subsequent step S4; if iterating through the "id" field, it is determined to be a basic string type, and the process proceeds to step S3.

[0058] In step S3, when a field is identified as a basic type, the system directly creates a leaf node and stores the field name and value in the leaf node. The leaf node marks the end of data parsing and no longer triggers recursion. For example, when processing the "id" field in business data, the system creates a leaf node with the field name "id" and stores the field value "S001" internally; similarly, when parsing "value": 25.6 in the sensors array, a leaf node is generated with the field name "value" and the field value "25.6".

[0059] In step S4, for the object type, all its key-value pairs can be recursively traversed: a child node is created for each key, and the value corresponding to the key is used as the input of the new node. Then, the system returns to step S2 to re-determine the data type. This step realizes the layer-by-layer decomposition of the hierarchical structure. For example, when parsing {"device": {"id": "S001", "sensors": [...]}}, the "device" field is determined to be an object type. The system traverses its key-value pairs: a child node is created for "id" and "S001" is stored (subsequently proceeding to step S3 due to its basic type), and a child node is created for "sensors" and an array value is passed in (proceeding to step S5 due to its array type), forming a structure where the "device" node has "id" and "sensors" child nodes attached.

[0060] In step S5, when the field is an array type, the system iterates through each element of the array, creates an independent child node for each element, and adds an index attribute to identify its position. The value of the element is used as the input for the new node, and the system returns to step S2 to continue parsing. The index attribute is a unique position identifier set for each array element's child node when the field in the business data is an array type. It records the element's sequential index (e.g., 0, 1, 2) in the original array, thus preserving the array's order in the tree structure and ensuring that the subsequent path selector can accurately locate data elements at specific index positions. For example, for the array fields [{"value": 25.6}, {"value": 65}] in the business data, a child node can be created for the first element with the index attribute index="0", and a child node can be created for the second element with the index attribute index="1". This allows the DOM tree to accurately restore the array's sequential relationship, avoiding data order loss due to structural flattening.

[0061] For example, when processing the `sensors` array, two child nodes are created: the first child node has an index of "0" and stores {"type": "temp", "value": 25.6}; the second child node has an index of "1" and stores {"type": "hum", "value": 65}. Since the elements in the array are objects, the value of each child node triggers step S4, which further breaks it down into "type" and "value" leaf nodes.

[0062] In step S6, the virtual root node is set as the root of the entire tree. Then, all nodes generated in steps S3, S4, and S5 are mounted according to their hierarchical relationship: leaf nodes are added to their parent object nodes, and array element nodes are added to the array parent node. This operation ensures that the hierarchical structure of the business data is completely mapped to a tree model. For example, based on the aforementioned business data, the virtual root node "root" mounts the "device" node; the "device" node mounts the "id" leaf node and the "sensors" array node; the "sensors" node mounts two indexed child nodes (index 0 and 1), and each child node then mounts the "type" and "value" leaf nodes. The final DOM tree accurately reproduces the original data structure: root→device→(id: S001, sensors→[0: {type: temp, value: 25.6}, 1: {type: hum, value: 65}]).

[0063] like Figure 2 As shown, the DOM tree is constructed based on the parsed field names, field values, and the hierarchical relationships between field names. It also includes: using the second language model, generating semantic tags corresponding to each node based on the semantic information of the field names and field values ​​in each node of the DOM tree; adding the semantic tags corresponding to each node to the corresponding node, and finally obtaining the DOM tree corresponding to the data acquisition device.

[0064] In the process of building the DOM tree, not only are the field names and values ​​of business data mapped to a tree-like node structure, but the nodes are also semantically enhanced by introducing a second major language model.

[0065] The second language model is a natural language processing model specifically designed for deep semantic understanding. Unlike the general-purpose first language model, the second language model is trained on domain-specific data and focuses on semantic parsing and classification tasks for device data fields. It can understand the terminology system of IoT-related professional fields such as industrial equipment and sensors, and accurately identify the true meaning of various abbreviations, aliases, and professional terms.

[0066] Specifically, after obtaining the basic DOM tree structure using a recursive traversal algorithm, the second major language model can be invoked to perform semantic analysis on each node in the DOM tree. This second major language model can comprehensively analyze the semantic information of field names and values ​​in each node, generating corresponding semantic tags. These semantic tags represent standardized concepts of fields in specific scenarios, providing a foundation for subsequent semantic-based data retrieval and transformation.

[0067] For example, the recursive traversal algorithm parses the data and constructs a basic DOM tree structure as follows: a virtual root node is attached to a "dev_stat" object node, which in turn attaches a "t_val" leaf node storing the value 26.5 and a "h_level" leaf node storing the value 65.2. The second language model is then used to perform semantic analysis on this basic DOM tree, identifying that the "t_val" node represents temperature and the "h_level" node represents humidity. The semantic tags obtained from the analysis are injected into the corresponding nodes to obtain a semantically enhanced DOM tree: a virtual root node is attached to a "dev_stat" node, and the "t_val" node attached to this node has the semantic tag "temperature" added in addition to the original data, while the "h_level" node has the semantic tag "humidity" added.

[0068] In this embodiment, the internal implementation structure of the second major language model is not limited. For example... Figure 2 The diagram illustrates an example of the internal implementation structure of a second-largest language model. In this example, the second-largest language model includes a semantic encoding layer, a feature extraction layer, a label generation layer, and a decision output layer. Based on the internal implementation of the second-largest language model, a method for generating semantic labels for each node using the semantic information of field names and values ​​in each node of the DOM tree includes: inputting the field names and values ​​of each node in the DOM tree into the semantic encoding layer for text vectorization to obtain semantic vectors for each node; inputting the semantic vectors for each node into the feature extraction layer for context feature extraction to obtain semantic feature vectors for each node; inputting the semantic feature vectors into the label generation layer for semantic classification to obtain at least one candidate semantic label and its confidence level for each node; and inputting at least one candidate semantic label and its confidence level into the decision output layer to select candidate semantic labels whose confidence levels meet a preset threshold as the semantic labels for each node.

[0069] The semantic encoding layer is responsible for converting the input text information into a machine-understandable numerical representation. It takes the field names and values ​​of DOM tree nodes as input text and performs text vectorization processing using techniques such as word embedding, transforming discrete words into continuous semantic vector representations rich in semantic information. For example, for the node field name "t" and field value "25.6", the semantic encoding layer can treat these two field names as text, combine them into "t: 25.6", and then vectorize them. The field name "t" is encoded as a vector [0.85, -0.12, 0.73, ...], and the field value "25.6" is encoded as [0.02, 0.98, -0.05, ...]. These are then concatenated or weighted and fused into the node's overall semantic vector [0.87, 0.86, 0.68, ...]. This vectorization allows subsequent feature extraction layers to perform further semantic analysis based on mathematical operations rather than string matching.

[0070] The feature extraction layer receives the semantic vector output by the semantic encoding layer and performs deeper contextual feature extraction using a deep neural network (e.g., an attention-based deep learning model like the Transformer Model or a Convolutional Neural Network). The feature extraction layer can identify the inherent relationship between field names and field values ​​and strengthen key features by incorporating domain knowledge, forming a more discriminative semantic feature vector. For example, following the above embodiment, the feature extraction layer can process [0.87, 0.86, 0.68,...] and combine it with patterns from historical data (e.g., "t" represents temperature in 90% of sensor scenarios) to output an enhanced semantic feature vector [0.91, 0.95, 0.32,...]. This semantic feature vector, by incorporating the industry context of temperature monitoring, significantly improves the accuracy of subsequent semantic label generation.

[0071] The label generation layer takes the semantic feature vector as input, performs semantic classification, and obtains at least one candidate semantic label and its confidence score for each node. The candidate semantic label is one or more standardized semantic concepts that the second language model identifies from a predefined label library based on the input semantic feature vector. Due to the complexity of natural language, such as polysemy and synonyms, a single field may correspond to multiple potential semantic interpretations; therefore, multiple candidate labels are generated for subsequent decision-making. The confidence score is a numerical rating between 0 and 1, used to quantify the second language model's certainty about the correctness of each candidate semantic label. A higher confidence score indicates that the second language model believes the semantic label matches the input field name and value more closely. The confidence score can be calculated using the softmax function, which transforms the raw scores output by the neural network into a probability distribution, ensuring that the sum of the confidence scores of all candidate labels is 1. For example, following the above embodiment, when the semantic feature vector is [0.91, 0.95, 0.32, ...], the label generation layer can output: "Candidate semantic label temperature, confidence 0.96", "Candidate semantic label voltage, confidence 0.02", "Candidate semantic label humidity, confidence 0.01", and "Candidate semantic label unknown, confidence 0.01". These candidate semantic labels cover the possible business meanings of the corresponding fields, and the confidence level quantifies the certainty of the second language model for each semantic label, providing a basis for subsequent decision-making.

[0072] The decision output layer filters candidate semantic labels based on preset threshold conditions to obtain the semantic labels corresponding to each node. The preset threshold conditions are quantitative standards or rules pre-set for filtering candidate labels during semantic label decision-making. In this embodiment, the specific form of the preset conditions is not limited and can be flexibly configured according to different application scenarios and accuracy requirements. For example, the preset threshold conditions can be a single confidence level preset: setting a fixed confidence score, such as 0.8. Only labels with a confidence level of 0.8 or higher among all candidate semantic labels will be ultimately adopted; or it can be a Top-N selection combined with the lowest confidence level: first selecting the top N labels with the highest confidence levels (such as Top-1 or Top-2), while requiring their confidence levels to be higher than an absolute lower limit (such as 0.6); or a dynamic relative threshold: instead of setting a fixed value, selecting other labels whose scores differ from the highest confidence label by a certain proportion or range, for example, selecting all candidate semantic labels with a confidence level not lower than 80% of the highest confidence label.

[0073] For example, following the above embodiment, if the preset threshold condition is to use semantic labels with a confidence level greater than 0.4 as the semantic labels of the corresponding nodes, then for the "candidate semantic label temperature, confidence level 0.96", "candidate semantic label voltage, confidence level 0.02", "candidate semantic label humidity, confidence level 0.01", and "candidate semantic label unknown, confidence level 0.01" output by the label generation layer, the decision output layer selects temperature as the semantic label of the node.

[0074] In one optional embodiment, the DOM tree cache corresponding to any data acquisition device pre-maintained by the IoT platform is initialized. The DOM tree cache is used to store the DOM trees currently loaded in memory. Each DOM tree is uniquely indexed by the corresponding data acquisition device identifier. When business data uploaded by the target data acquisition device is received, the identifier of the target data acquisition device is obtained, and the DOM tree cache is queried to see if a corresponding target DOM tree exists based on the data acquisition device identifier. If the query result is empty, the DOM tree corresponding to the target data acquisition device is loaded from persistent storage and added to the DOM tree cache, while the initial usage time of the DOM tree is recorded. If the query result is not empty, the target DOM tree is used directly, and the usage time of the target DOM tree in the cache is updated.

[0075] Initializing the DOM tree cache refers to preparing the runtime environment for the pre-established DOM tree management mechanism when the IoT platform starts up. This cache is essentially a high-efficiency storage area in memory, specifically used to store the DOM trees corresponding to currently active data acquisition devices. By storing the DOM trees in memory, the need to reload and parse from slow storage is avoided every time device data is processed, thereby significantly improving the response speed of data processing and the overall performance of the IoT platform.

[0076] In this context, the unique indexing of each DOM tree through its corresponding data acquisition device identifier means that the IoT platform assigns a unique retrieval key to each DOM tree. This key is bound to a specific data acquisition device, allowing for quick location of the DOM tree corresponding to the target data acquisition device in the cache. For example, the data acquisition device identifier could be the device's serial number at the time of manufacture (e.g., SN20240820001), a unique ID (IdentityDocument) assigned by the IoT platform during device registration (e.g., DEVICE_ZJ_TEMP_001), or the data acquisition device's MAC address (Media Access Control Address), IMEI number (International Mobile Equipment Identity), etc. This embodiment does not impose any limitations on these aspects.

[0077] When business data uploaded by the target data acquisition device is received, the identifier of the target data acquisition device is obtained, and the corresponding target DOM tree is queried in the DOM tree cache based on the identifier.

[0078] If the query result is empty, the DOM tree corresponding to the target data acquisition device is loaded from persistent storage and added to the DOM tree cache. The initial usage time of the DOM tree is also recorded. Persistent storage is a non-volatile storage medium used for long-term data preservation. For example, persistent storage can be a relational database, document database, distributed file system, or local disk file, etc., and this embodiment is not limited to any particular type. When the required DOM tree is not present in the cache, the IoT platform can load the corresponding DOM tree from these persistent storage sources into memory.

[0079] If the query result is not empty, the target DOM tree is used directly, and the usage time of the target DOM tree in the cache is updated. This means that when the target data collection device repeatedly uploads business data, the IoT platform does not need to reload the DOM tree, but only needs to update the most recent access time of the DOM tree in the cache to ensure that it is not prematurely replaced.

[0080] For example, the IoT platform receives business data from an industrial temperature and humidity sensor (device identifier: Sensor_2024-Temp) as the target data acquisition device.

[0081] During the initialization phase: When the IoT platform starts, a DOM tree cache is created, and the DOM trees of high-frequency devices are preloaded. However, Sensor_2024-Temp has not yet been connected, and there is no corresponding target DOM tree in the cache. For the first business data upload: {"t": 25.6}, the platform obtains the target data acquisition device identifier Sensor_2024-Temp. If the cache query shows no match (empty result), the platform loads the DOM tree (structure: root→ data→ t) of the target data acquisition device from persistent storage (such as the MySQL table device_dom_trees), stores it in the cache, and records the initial usage time as 2024-06-01T12:00:00. If the cache query finds a match for the DOM tree corresponding to Sensor_2024-Temp, the platform directly uses that DOM tree to parse the data and updates the usage time to 2024-06-01T12:00:05.

[0082] In the above embodiments, by loading the DOM tree and dynamically updating the cache usage time only when the target data acquisition device is first connected, the frequency of access to persistent storage is reduced, and the resource utilization of the IoT platform is optimized. In scenarios where the target data acquisition device reports frequently, this mechanism ensures the high throughput and real-time response capability of the IoT platform and supports the seamless access of new data acquisition devices, thereby significantly improving the efficiency and scalability of the IoT platform in processing heterogeneous data without reducing flexibility.

[0083] It should be noted that in the above embodiments, a large language model refers to a model whose parameters meet the set parameter quantity requirements, and the parameter quantity requirements are not limited. Different definitions may exist in different scenarios and domains. For example, in some scenarios or domains, a large language model refers to a model with a parameter scale reaching tens of billions, hundreds of billions, or even trillions; this definition of the model's parameter quantity is merely an example.

[0084] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0085] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0086] Figure 3 This is a schematic diagram of an electronic device structure provided for an exemplary embodiment of this application. For example... Figure 3 As shown, the electronic device includes: a memory 34 and a processor 35.

[0087] Memory 34 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, field names to be converted used by the target data acquisition device, the DOM tree, and field values ​​corresponding to the field names to be converted, etc.

[0088] The memory 34 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0089] Processor 35, coupled to memory 34, is used to execute a computer program in memory 34 for: a data processing method applied to an Internet of Things (IoT) platform, wherein the IoT platform pre-maintains a conversion relationship between field names used by the IoT platform and field names used by various data acquisition devices, and pre-constructs multiple DOM trees corresponding to the various data acquisition devices. Multiple nodes in the DOM trees store the field names and field values ​​used by the corresponding data acquisition devices. The method includes: receiving business data uploaded by a target data acquisition device, the business data including field names used by the target data acquisition device and to be converted; determining a target DOM tree corresponding to the target data acquisition device from the multiple DOM trees; generating a path selector based on the field names to be converted, and using the path selector to determine a target node from the target DOM tree, the target node being the node containing the field names to be converted; obtaining the field values ​​corresponding to the field names to be converted from the target node; and converting the field values ​​corresponding to the field names to be converted into field values ​​under the field names used by the IoT platform based on the conversion relationship corresponding to the target data acquisition device, so that the IoT platform can perform data processing based on the converted field names and field values.

[0090] In an optional embodiment, the nodes on the DOM tree also include semantic tags corresponding to field names and field values; the processor 35 uses a path selector to determine the target node from the target DOM tree, including: calling the first language model to perform semantic parsing on the field name to be converted to obtain the semantic information to be queried; using the path selector, performing semantic traversal on the target DOM according to the semantic information to be queried and the semantic tags of each node in the target DOM tree to obtain a set of candidate nodes; constructing a subtree according to the hierarchical relationship between each node in the candidate node set; using the path selector, traversing on the subtree according to the field name to be converted to obtain the target node including the field name to be converted.

[0091] In an optional embodiment, the processor 35 receives business data uploaded by each data acquisition device. The business data includes field names and field values ​​used by the corresponding data acquisition device. The processor 35 parses the field names and field values ​​in the business data based on a recursive traversal algorithm and constructs a DOM tree based on the parsed field names, field values, and the hierarchical relationships between field names. The field names and field values ​​correspond to nodes in the DOM tree, and the hierarchical relationships between field names correspond to the hierarchical relationships between nodes in the DOM tree.

[0092] In an optional embodiment, the processor 35 parses the field names and field values ​​in the business data based on a recursive traversal algorithm, and constructs a DOM tree according to the parsed field names, field values, and the hierarchical relationships between field names. This includes: based on the data type of the field values ​​in the business data, performing the following recursive traversal operations: S1. Create a virtual root node and use it as the starting node for the recursive traversal; S2. For the field in the currently traversed business data, determine whether the data type of the current field is a basic type; S3. If the data type of the current field is a basic type, create a leaf node and store the field name and field value in the leaf node; S4. If the data type of the field... If the type is an object, iterate through the key-value pairs contained in the field, create a corresponding child node for each key, and store the value corresponding to each key in the corresponding child node, then return to step S2; S5. If the data type of the field is an array, iterate through each element in the array contained in the field, create a corresponding child node for each element, set an index attribute in the child node to mark the position of each element in the array, and store the value of each element in the corresponding child node, then return to step S2; S6. Use the virtual root node as the root node of the DOM tree, and add the child nodes generated by steps S3, S4 or S5 to the child node list of their respective parent nodes.

[0093] In an optional embodiment, the processor 35 constructs a DOM tree based on the parsed field names, field values, and the hierarchical relationships between field names, and further includes: using a second language model to generate semantic tags corresponding to each node based on the semantic information of the field names and field values ​​in each node of the DOM tree; Add the semantic tags corresponding to each node to the corresponding node.

[0094] In one optional embodiment, the second large language model includes a semantic encoding layer, a feature extraction layer, a label generation layer, and a decision output layer. The processor 35 utilizes the second large language model to generate semantic labels corresponding to each node based on the semantic information of the field names and values ​​in each node of the DOM tree. This includes: inputting the field names and values ​​of each node in the DOM tree into the semantic encoding layer for text vectorization to obtain semantic vectors corresponding to each node; inputting the semantic vectors corresponding to each node into the feature extraction layer for context feature extraction to obtain semantic feature vectors corresponding to each node; inputting the semantic feature vectors into the label generation layer for semantic classification to obtain at least one candidate semantic label and its confidence level for each node; and inputting at least one candidate semantic label and its confidence level into the decision output layer to select candidate semantic labels whose confidence levels meet a preset threshold condition as the semantic labels corresponding to each node.

[0095] In an optional embodiment, the processor 35 initializes a DOM tree cache corresponding to any data acquisition device pre-maintained by the IoT platform. The DOM tree cache is used to store the DOM trees currently loaded in memory. Each DOM tree is uniquely indexed by the corresponding data acquisition device identifier. When business data uploaded by the target data acquisition device is received, the identifier of the target data acquisition device is obtained, and the DOM tree cache is queried to see if a corresponding target DOM tree exists based on the target data acquisition device identifier. If the query result is empty, the DOM tree corresponding to the target data acquisition device is loaded from persistent storage and added to the DOM tree cache, while the initial usage time of the DOM tree is recorded. If the query result is not empty, the target DOM tree is used directly, and the usage time of the target DOM tree in the cache is updated.

[0096] Furthermore, such as Figure 3 As shown, the electronic device also includes other components such as a communication component 36, a display 37, a power supply component 38, and an audio component 39. Figure 3 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 3 The components shown.

[0097] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by an electronic device in the above method embodiments.

[0098] The above Figure 3 The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0099] The above Figure 3 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0100] The above Figure 3 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0101] The above Figure 3 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0102] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0106] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0107] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0108] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0109] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0110] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data processing method, characterized in that, Applied to an IoT platform, the IoT platform pre-maintains the conversion relationship between field names used by the IoT platform and field names used by various data acquisition devices, and pre-constructs multiple DOM trees corresponding to various data acquisition devices. Multiple nodes in the DOM trees store the field names and field values ​​used by the corresponding data acquisition devices. The method includes: Receive service data uploaded by the target data acquisition device, wherein the service data includes field names used by the target data acquisition device and to be converted; From the plurality of DOM trees, determine the target DOM tree corresponding to the target data acquisition device; Based on the field name to be converted, a path selector is generated, and the path selector is used to determine the target node from the target DOM tree. The target node is the node where the field name to be converted is located. From the target node, obtain the field value corresponding to the field name to be converted; Based on the conversion relationship corresponding to the target data acquisition device, the field value corresponding to the field name to be converted is converted into the field value under the field name used by the IoT platform, so that the IoT platform can perform data processing based on the converted field name and field value.

2. The method according to claim 1, characterized in that, The nodes in the DOM tree also include semantic tags corresponding to field names and field values; Determining the target node from the target DOM tree using the path selector includes: The first major language model is invoked to perform semantic parsing on the field names to be converted, thereby obtaining the semantic information to be queried; Using the path selector, semantic traversal is performed on the target DOM based on the semantic information to be queried and the semantic tags of each node in the target DOM tree to obtain a set of candidate nodes; Construct a subtree based on the hierarchical relationship between nodes in the candidate node set; The path selector traverses the subtree according to the field name to be converted to obtain the target node that includes the field name to be converted.

3. The method according to claim 1, characterized in that, Also includes: Receive business data uploaded by each data acquisition device, wherein the business data includes the field names and field values ​​used by the corresponding data acquisition device; The recursive traversal algorithm is used to parse the field names and values ​​in the business data, and a DOM tree is constructed based on the parsed field names, their corresponding field values, and the membership relationships between the field names. In this context, field names and field values ​​correspond to nodes in the DOM tree, and the membership relationships between field names correspond to the hierarchical relationships between nodes in the DOM tree.

4. The method according to claim 3, characterized in that, The recursive traversal algorithm is used to parse the field names and values ​​in the business data, and a DOM tree is constructed based on the parsed field names, their corresponding values, and the hierarchical relationships between the field names, including: Based on the data type of the field values ​​in the business data, perform the following recursive traversal operation: S1. Create a virtual root node and use the virtual root node as the starting node for recursive traversal; S2. For the fields in the currently traversed business data, determine whether the data type of the current field is a basic type; S3. If the data type of the current field is a basic type, then create a leaf node and store the field name and field value of the field in the leaf node; S4. If the data type of the field is an object type, then traverse the key-value pairs contained in the field, create a corresponding child node for each key, and store the value corresponding to each key into the corresponding child node, and return to execute step S2. S5. If the data type of the field is an array type, then traverse each element in the array contained in the field, create a corresponding child node for each element, set an index attribute in the child node to mark the position of each element in the array, and store the value of each element in the corresponding child node, and return to execute step S2. S6. Use the virtual root node as the root node of the DOM tree, and add the child nodes generated by steps S3, S4 or S5 to the child node list of their respective parent nodes.

5. The method according to any one of claims 3-4, characterized in that, The step of constructing the DOM tree based on the parsed field names and their corresponding field values, as well as the hierarchical relationships between the field names, also includes: Using the second major language model, semantic tags are generated for each node based on the semantic information of the field names and values ​​in each node of the DOM tree. Add the semantic tags corresponding to each node to the corresponding node.

6. The method according to claim 5, characterized in that, The second major language model includes a semantic encoding layer, a feature extraction layer, a label generation layer, and a decision output layer. Using this second major language model, semantic labels corresponding to each node are generated based on the semantic information of the field names and values ​​in each node of the DOM tree, including: The field names and values ​​of each node in the DOM tree are input into the semantic encoding layer for text vectorization processing to obtain the semantic vectors corresponding to each node. The semantic vectors corresponding to each node are input into the feature extraction layer to extract context features and obtain the semantic feature vectors corresponding to each node. The semantic feature vector is input into the label generation layer for semantic classification processing to obtain at least one candidate semantic label and its confidence level for each node. The at least one candidate semantic label and its confidence level are input into the decision output layer, and candidate semantic labels whose confidence levels meet the preset threshold conditions are selected as the semantic labels corresponding to each node.

7. The method according to claim 1, characterized in that, Also includes: Initialize the DOM tree cache corresponding to any data acquisition device that the IoT platform has pre-maintained. The DOM tree cache is used to store the DOM tree currently loaded in memory. Each DOM tree is uniquely indexed by the corresponding data acquisition device identifier. When receiving service data uploaded by the target data acquisition device, the identifier of the target data acquisition device is obtained, and the DOM tree cache is queried to see if a corresponding target DOM tree exists based on the identifier of the target data acquisition device. If the query result is empty, the DOM tree corresponding to the target data acquisition device is loaded from the persistent storage, and the DOM tree is added to the DOM tree cache. At the same time, the initial usage time of the DOM tree is recorded. If the query result is not empty, the target DOM tree is used directly, and the usage time of the target DOM tree in the cache is updated.

8. An electronic device, characterized in that, include: A memory and a processor, the memory for storing a computer program, the processor coupled to the memory for executing the computer program to implement the steps of the method of any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-7.

10. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-7.

Images