Multi-label data instance cleaning method and device applied to the Internet of Things

Abstract

The present application provides a multi-label data instance cleaning method and apparatus for the Internet of Things. In the present application, based on partial label learning and taking into account the contribution of candidate labels, as well as the similarity of labels between data instances of the same type and the difference of labels between data instances of different types, the candidate labels of the target data instances are cleaned, so as to select a target label corresponding to each target data instance from at least two candidate labels of each target data instance, and finally realize the noise label cleaning of at least two candidate labels of the target data instance to obtain the final target label.

Description

Technical Field

[0001] The present application relates to Internet of Things technology, and in particular to a method and device for cleaning multi-label data instances applied to the Internet of Things. Background Art

[0002] In IoT applications, such as those with a cloud-edge-end or edge-end architecture, each IoT device, whether deployed as an "edge" or an "end," generates at least one piece of data (also referred to as a data instance). IoT devices deployed as "edges" are referred to as edge devices, such as those with intelligent algorithms capable of intelligent processing and / or capable of interacting with the cloud. IoT devices deployed as "ends" are referred to as terminal devices, such as cameras and other devices with data collection capabilities.

[0003] When data instances are labeled using machine algorithms and / or manual labeling, it's common for the same data instance to be assigned at least two completely different labels. However, having at least two completely different labels on the same data instance can affect subsequent IoT applications based on that data instance, such as model training. Summary of the Invention

[0004] Embodiments of the present application provide a multi-label data instance cleaning method and apparatus for the Internet of Things, so as to clean at least two labels of the same data instance based on partial label learning.

[0005] The present application provides a method for cleaning multi-label data instances applied to the Internet of Things. The method is applied to an Internet of Things device and includes:

[0006] Obtaining a target data instance to be cleaned; each target data instance is labeled with at least two candidate labels;

[0007] The candidate labels of the target data instances are cleaned according to the candidate label contribution weights and specified principles by a trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels of each target data instance; the candidate label contribution weights are used to indicate the contribution of the candidate labels of the target data instances in determining the target label corresponding to the target data instance;

[0008] The designation principle is as follows: different target data instances marked with completely different candidate tags correspond to different target tags, and different target data instances marked with at least one identical candidate tag and satisfying similar conditions correspond to the same target tag.

[0009] The embodiment of the present application provides a multi-label data instance cleaning device applied to the Internet of Things, the device being applied to an Internet of Things device, including:

[0010] An obtaining unit, configured to obtain a target data instance to be cleaned; each target data instance is labeled with at least two candidate labels;

[0011] a cleaning unit, configured to clean the candidate labels labeled with the target data instances according to the candidate label contribution weights and a specified principle using a trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels labeled with each target data instance; the candidate label contribution weights are used to indicate the contribution of the candidate labels labeled with the target data instances in determining the target label corresponding to the target data instance;

[0012] The designation principle is as follows: different target data instances marked with completely different candidate tags correspond to different target tags, and different target data instances marked with at least one identical candidate tag and satisfying similar conditions correspond to the same target tag.

[0013] An embodiment of the present application further provides an electronic device. The electronic device includes: a processor and a machine-readable storage medium;

[0014] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0015] The processor is used to execute machine-executable instructions to implement the steps of the above-disclosed method.

[0016] It can be seen from the above technical solution that this embodiment is based on a partial label learning target data cleaning model. The candidate labels marked with the target data instance are cleaned through the target data cleaning model to select a target label corresponding to each target data instance from at least two candidate labels marked with each target data instance. Finally, the noise label cleaning of at least two candidate labels marked with the target data instance is realized to obtain the final target label.

[0017] Furthermore, in this embodiment, when cleaning the candidate labels of the target data instances using the target data cleaning model, both the contribution of the candidate labels and the similarity of labels between the same type of data instances and the difference of labels between different types of data instances are considered, which improves the accuracy of the final determined target labels.

[0018] Furthermore, this embodiment is applied to IoT devices close to the data source, and there is no need to transmit data to the cloud or other third-party devices, which improves data cleaning efficiency and saves communication transmission resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 A flow chart of the method provided in the embodiment of the present application;

[0021] Figure 2 A flowchart for determining whether different target data instances meet similarity conditions provided in an embodiment of the present application;

[0022] Figure 3 Model training flow chart provided for the embodiment of this application;

[0023] Figure 4 A diagram of the device structure provided in an embodiment of the present application;

[0024] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0026] The terms used in this application are for the purpose of describing particular embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0027] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0028] See also Figure 1 , Figure 1A flowchart of the method provided in an embodiment of the present application. This process is applied to an IoT device. The IoT device here has the ability to communicate through local networking and also has the ability to communicate with external networks. Taking the application in an IoT environment with an "edge-end" structure as an example, the IoT device here can be an edge device deployed as an "edge" in the IoT, or the IoT device here can be a terminal device deployed as an "end" in the IoT.

[0029] like Figure 1 As shown, the process may include the following steps:

[0030] Step 101: Obtain target data instances to be cleaned; each target data instance is labeled with at least two candidate tags.

[0031] In an IoT environment, such as the aforementioned "edge-end" IoT environment, IoT devices deployed as "edges" or "ends" generate data (also known as data instances) for local decision-making based on requirements, such as malware detection. A single piece of data can be referred to as a data instance, or multiple pieces of data with the same attributes, such as malware detection, can also be referred to as a data instance. This embodiment does not specifically limit the specific implementation of data instances.

[0032] Each data instance can be labeled using a machine algorithm or manually. In one example, the same data instance is often labeled with at least two labels. In this case, the data instance labeled with at least two labels can be referred to as the target data instance to be cleansed, and the labels assigned to the target data instance are referred to as candidate labels. This ultimately achieves the goal of obtaining the target data instance to be cleansed in step 101.

[0033] In step 102, the candidate labels of the target data instances are cleaned according to the candidate label contribution weights and specified principles through the trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels of each target data instance.

[0034] In this embodiment, among the at least two candidate labels for the target data instance, only one label is the true label of the target data instance (i.e., the target label). Based on this, in this embodiment, the target data cleaning model is designed based on partial label learning to clean the target data instances labeled with at least two candidate labels.

[0035] In this embodiment, when noise labels are cleaned for target data instances marked with at least two candidate labels through the target data cleaning model, both the contribution of the candidate labels and the similarity of labels between data instances of the same type and the difference of labels between data instances of different types are considered.

[0036] Here, the contribution of a candidate tag is represented by its contribution weight. This weight indicates the contribution of the candidate tag associated with the target data instance in determining the target tag for that instance. This weight effectively accounts for the contribution of the candidate tags, helping to ensure the accuracy of the final target tag.

[0037] The similarity of labels between data instances of the same class and the difference in labels between data instances of different classes are characterized by a specified principle. In one example, the specified principle can be defined as follows: different target data instances labeled with completely different candidate labels correspond to different target labels, while different target data instances labeled with at least one identical candidate label and meeting the similarity condition correspond to the same target label.

[0038] Based on the above description, finally as described in step 102, the candidate labels labeled with the target data instances are cleaned by the trained target data cleaning model based on partial labeling learning, taking into account the contribution weights of the candidate labels and the above-mentioned specified principles, so as to select a target label corresponding to each target data instance from at least two candidate labels labeled with each target data instance.

[0039] So far, completed Figure 1 The process shown.

[0040] pass Figure 1 As can be seen from the shown process, this embodiment is based on a partial labeling learning target data cleaning model, through which the candidate labels marked by the target data instances are cleaned, so as to select a target label corresponding to each target data instance from at least two candidate labels marked by each target data instance, and finally realizes the noise label cleaning of at least two candidate labels marked by the target data instance to obtain the final target label.

[0041] Furthermore, in this embodiment, when cleaning the candidate labels of the target data instances using the target data cleaning model, both the contribution of the candidate labels and the similarity of labels between the same type of data instances and the difference of labels between different types of data instances are considered, which improves the accuracy of the final determined target labels.

[0042] Furthermore, this embodiment is applied to IoT devices close to the data source, and there is no need to transmit data to the cloud or other third-party devices, which improves data cleaning efficiency and saves communication transmission resources.

[0043] In this embodiment, the different target data instances that meet the similar conditions can be Figure 2 The process shown is implemented as follows: Figure 2 As shown, the process may include:

[0044] Step 201: for the i-th target data instance x i , determine the i-th target data instance x i The kth nearest neighbor x ik ;

[0045] Here, the i-th target data instance x i Can generally refer to any target data instance.

[0046] Optionally, in step 201, the i-th target data instance x may be determined by a K-Nearest Neighbor (KNN) classification algorithm. i The kth nearest neighbor x ik The core of the K-nearest neighbor classification algorithm is: in the feature space, if most of the k nearest (i.e., the closest) samples near a sample belong to a certain category, then the sample also belongs to this category. As for how to use the K-nearest neighbor (KNN: k-Nearest Neighbor) classification algorithm to determine the i-th target data instance x i The kth nearest neighbor x ik , we will not describe it in detail here, please refer to the K nearest neighbor algorithm for details.

[0047] Step 202: Based on the kth nearest neighbor x ik , and according to the semantic relevance between the i-th target data instance and the j-th target data instance, determine the similarity S between the i-th target data instance and the j-th target data instance ij .

[0048] Optionally, in this embodiment, step 202 may be implemented by the following formula 1:

[0049]

[0050] In Formula 1, i represents the i-th target data instance, j represents the j-th target data instance, and x i represents the feature vector of the i-th target data instance, x j represents the feature vector of the jth target data instance, m is the total number of target data instances, x ik Represents xi The kth nearest neighbor of .

[0051] Step 203, when S ij is greater than or equal to the preset threshold, then it is determined that the i-th target data instance and the j-th target data instance are similar. ij If the value is less than a preset threshold, it is determined that the i-th target data instance and the j-th target data instance are not similar.

[0052] In this embodiment, the preset threshold is less than 1, such as 0.5. In the application, when S ij If it is greater than or equal to the preset threshold and is closer to 1, it means that the i-th target data instance and the j-th target data instance are more similar. Conversely, the i-th target data instance and the j-th target data instance are not similar.

[0053] So far, completed Figure 2 The process shown.

[0054] In a specific embodiment, Figure 2 The process shown can be specifically implemented by a model parameter (denoted as a matrix R) designed through learning in the target data cleaning model. For example, the i-th target data instance (including the labeled candidate labels) and the j-th target data instance (including the labeled candidate labels) are input into the target data cleaning model. If the matrix R in the target data cleaning model outputs a first value, such as 1, it means that the i-th target data instance and the j-th target data instance have completely different candidate labels. Conversely, if the matrix R in the target data cleaning model outputs a second value, such as 0, it means that the i-th target data instance and the j-th target data instance have at least one identical candidate label.

[0055] Below through Figure 3 Describe how to train the target data cleaning model:

[0056] See also Figure 3 , Figure 3 This is a flow chart of the model training provided in the embodiment of this application. Figure 3 As shown, the process may include the following steps:

[0057] Step 301: Obtain the contribution weight of each candidate label of the training data instance in the training set.

[0058] In this embodiment, the training data instances in the training set may have multi-dimensional features. For example, the training data instances may have 784-dimensional features. For each training data instance, when the training data instance is labeled with two or more candidate labels, one of the candidate labels is the target label for the training data instance.

[0059] This embodiment considers the contribution of all candidate labels when training the model. In a specific implementation, the different confidence levels at which the candidate labels are labeled can be formalized as a potential label distribution, which can be represented by the contribution weights of the candidate labels. Here, the contribution weight representation is used to indicate the contribution of the candidate label in determining the target label corresponding to the training data instance. As an embodiment, the contribution weights of the candidate labels here can be pre-configured, or they can be obtained by optimizing the contribution weights of the pre-configured candidate labels according to a specified optimization algorithm. But in any case, the contribution weights of each candidate label labeled for the training data instance in the training set can eventually be obtained as described in step 301. In an example, the above-mentioned specified optimization algorithm can be a Lagrangian function optimization algorithm, which is not limited here.

[0060] In step 302 , a target data cleaning model is trained according to preset principles and contribution weights of candidate labels of each training data instance.

[0061] In one example, training the target data cleaning model in step 302 according to preset principles and contribution weights of candidate labels of each training data instance may include:

[0062] In step a1, a reference data cleaning model is trained according to preset principles and the contribution weights of the candidate labels of each training data instance.

[0063] The preset principle here is set based on maximizing the similarity of labels between data instances of the same type and the difference of labels between data instances of different types. Specifically, the preset principle requires that training data instances labeled with completely different candidate labels correspond to different target labels, and training data instances labeled with at least one identical candidate label and meeting similar conditions correspond to the same target label.

[0064] In this embodiment, the trained reference data cleaning model has linear separable parameters. Here, the linear separable parameters are used to solve the linear inseparability between different data instances input to the model. In one example, the linear separable parameters are obtained by mapping the training data instances into a high-dimensional space. The linear separable parameters are related to the contribution weights of the candidate labels of each training data instance. The following formula

[0065] Equation 2 shows an example of linearly separable parameters:

[0066]

[0067] In formula 2, W represents the linear separability parameter, Represents the instance obtained by mapping the training data instance to the high-dimensional space, K represents the configured Gaussian kernel function, and A represents the contribution weight of each candidate label for each training data instance.

[0068] According to Formula 2, when a data instance is input into the model, Expressed as KA, there is no need to solve the high-dimensional It can be calculated directly using K and A, which solves the linear inseparability problem of the data instances subsequently input into the model. Moreover, through W, the contribution weight of the candidate labels is taken into account when the candidate labels of the target data instances are subsequently cleaned by the target data cleaning model.

[0069] In this embodiment, K may be a commonly used Gaussian kernel, for example, K is represented by the following formula:

[0070] K ij =exp(-‖x i -x j ‖)^2 / 2*σ^2) (Formula 3)

[0071] In formula 3, x i 、x j As described above, σ is the smoothing parameter that has been set.

[0072] The reference data cleaning model is described above.

[0073] Step a2: testing the reference data cleaning model using test data instances in the test set. When the reference data cleaning model meets preset test conditions, the reference data cleaning model is determined as the target data cleaning model.

[0074] Of course, when the reference data cleaning model does not meet the preset test conditions, the model parameters are adjusted, and the model is trained again by returning to step a1 based on the adjusted model parameters.

[0075] Finally, through Figure 3 The process shown trains the target data cleaning model.

[0076] The following describes how to test the reference data cleaning model using test data instances in the test set:

[0077] Specifically, in this embodiment, testing the reference data cleaning model using the test data instances in the test set may include:

[0078] In step b1, the Gaussian function value between each test data instance and each training data instance is calculated using the reference data cleaning model to obtain a Gaussian function value matrix. Gaussian function values ​​in the same row of the Gaussian function value matrix correspond to the same test data instance, and Gaussian function values ​​in different rows correspond to different test data instances.

[0079] For example, assuming there are m sets of training data instances and n sets of test data instances, the final Gaussian function value matrix is ​​m*n. For ease of description, the Gaussian function value matrix here can be denoted as kt.

[0080] In step b2, a prediction label matrix is ​​obtained by performing a specified operation on the Gaussian function value matrix and the obtained optimization weight matrix through a reference data cleaning model.

[0081] The optimized weight matrix is ​​obtained by optimizing the contribution weights of each candidate label for each training data instance in the training set according to a specified optimization algorithm. The specified optimization algorithm here can be a Lagrangian function optimization algorithm. As for how to optimize the contribution weights of each candidate label using the Lagrangian function optimization algorithm, we will not describe it in detail here. For more details, please refer to the Lagrangian function optimization algorithm.

[0082] Taking the Gaussian function value matrix kt as an example and the obtained optimized weight matrix A as an example, step b2 can be: multiply the optimized weight matrix A by the Gaussian function value matrix kt on the right to obtain the predicted label matrix, where the predicted label matrix can be kt*A. The predicted label matrix contains the predicted labels corresponding to the test data instances. As described above, the Gaussian function values ​​on the same row in the Gaussian function value matrix correspond to the same test data instance. In this embodiment, for the predicted label matrix, the label with the largest value in each row can be determined as the predicted label corresponding to the test data instance corresponding to that row.

[0083] Step b3, determining whether the reference data cleaning model meets the preset test conditions based on the comparison result between the predicted label of the test data instance in the predicted label matrix and the real label configured for the test data instance; wherein, when the predicted label of the test data instance is similar to the real label configured for the test data instance, the comparison result for the test data instance is a match; otherwise, the comparison result is a mismatch.

[0084] In this embodiment, for each test data instance, when the predicted label corresponding to the test data instance in the predicted label matrix is ​​similar to or consistent with the true label configured for the test data instance, the comparison result of the test data instance is determined to be a match; otherwise, the comparison result of the test data instance is determined to be a mismatch.

[0085] When the comparison results for more than a specified number of test data instances in the test set are matched, or when the comparison results for more than a specified proportion, such as 95%, of the test data instances in the test set are matched, the reference data cleaning model is determined to meet the preset test conditions. Otherwise, the reference data cleaning model is determined to not meet the preset test conditions. When the reference data cleaning model does not meet the preset test conditions, as described above, the model parameters are adjusted, and the model is trained again based on the adjusted model parameters in step a1.

[0086] The target data cleaning model has been described above. Once trained, it can be deployed on IoT devices, such as edge devices deployed as "edges" in the IoT or terminal devices deployed as "ends" in the IoT. This allows for accurate and efficient cleaning of candidate labels for target data instances, selecting a target label for each target data instance from at least two candidate labels.

[0087] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application:

[0088] See also Figure 4 , Figure 4 This is a diagram of the device structure provided in the embodiment of this application. The device is applied to Internet of Things devices. Figure 4 As shown, the device may include:

[0089] An obtaining unit, configured to obtain a target data instance to be cleaned; each target data instance is labeled with at least two candidate labels;

[0090] a cleaning unit, configured to clean the candidate labels of the target data instances according to a specified principle using a trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels of each target data instance;

[0091] The designation principle is as follows: different target data instances marked with completely different candidate tags correspond to different target tags, and different target data instances marked with at least one identical candidate tag and satisfying similar conditions correspond to the same target tag.

[0092] Optionally, the different target data instances satisfying the similarity condition include:

[0093] For the i-th target data instance x i , determine the i-th target data instance x i The kth nearest neighbor x ik ;

[0094] According to the kth nearest neighbor x ik , and according to the semantic relevance between the i-th target data instance and the j-th target data instance, determine the similarity S between the i-th target data instance and the j-th target data instance ij

[0095] When S ij is greater than or equal to the preset threshold, then it is determined that the i-th target data instance and the j-th target data instance are similar.ij If the value is less than a preset threshold, it is determined that the i-th target data instance and the j-th target data instance are not similar.

[0096] Optionally, the target data cleaning model is trained in the following manner:

[0097] Obtaining a contribution weight of each candidate label for a training data instance in a training set; wherein the training data instance in the training set is labeled with at least one candidate label, wherein, when a training data instance is labeled with two or more candidate labels, one of the candidate labels is a target label for the training data instance; the contribution weight of the candidate label is used to indicate the contribution of the candidate label in determining the target label corresponding to the training data instance;

[0098] The target data cleaning model is trained according to a preset principle and the contribution weights of the candidate labels of each training data instance; the preset principle requires that training data instances labeled with completely different candidate labels correspond to different target labels, and training data instances labeled with at least one identical candidate label and meeting similar conditions correspond to the same target label;

[0099] Among them, the trained target data cleaning model has linear separable parameters, which are obtained by mapping the training data instances to a high-dimensional space. The linear separable parameters are related to the contribution weights of each candidate label marked with each training data instance. The linear separable parameters are used to achieve linear separability of different target data instances input into the target data cleaning model.

[0100] Optionally, the linear separable parameter is expressed by the following formula:

[0101]

[0102] Wherein, W represents the linear separable parameter, Represents the instance obtained by mapping the training data instance to the high-dimensional space, K represents the configured Gaussian kernel function, and A represents the contribution weight of each candidate label for each training data instance.

[0103] Optionally, obtaining the contribution weight of each candidate label of the training data instance in the training set includes:

[0104] Obtain the contribution weight of each candidate label for the training data instances in the pre-configured training set; or,

[0105] Obtain the result obtained by optimizing the contribution weights of each candidate label of the training data instances in the pre-configured training set according to the specified optimization algorithm.

[0106] Optionally, training the target data cleaning model according to preset principles and contribution weights of candidate labels of each training data instance includes:

[0107] A reference data cleaning model is trained according to preset principles and the contribution weights of each candidate label of each training data instance;

[0108] The reference data cleaning model is tested by using a test data instance in a test set, and when the reference data cleaning model meets a preset test condition, the reference data cleaning model is determined as the target data cleaning model.

[0109] Optionally, the testing the reference data cleaning model using a test data instance in a test set includes:

[0110] Calculating the Gaussian function value between each test data instance and each training data instance by referring to the data cleaning model to obtain a Gaussian function value matrix; Gaussian function values ​​on the same row of the Gaussian function value matrix correspond to the same test data instance, and Gaussian function values ​​on different rows correspond to different test data instances;

[0111] A prediction label matrix is ​​obtained by performing a specified operation on the Gaussian function value matrix and the obtained optimization weight matrix using a reference data cleaning model; the optimization weight matrix is ​​obtained by optimizing the contribution weights of each candidate label labeled with the training data instances in the training set according to a specified optimization algorithm; the prediction label matrix includes the predicted labels corresponding to the test data instances;

[0112] Whether the reference data cleaning model meets the preset test conditions is determined based on the comparison result of the predicted label of the test data instance in the predicted label matrix and the real label configured for the test data instance; wherein, when the predicted label of the test data instance is similar to the real label configured for the test data instance, the comparison result for the test data instance is a match; otherwise, the comparison result is a mismatch.

[0113] Optionally, the IoT device is an edge device deployed as an “edge” in the IoT, or,

[0114] The IoT device is a terminal device deployed as an "end" in the IoT.

[0115] So far, completed Figure 4 Structural description of the device shown.

[0116] The present application also provides Figure 4 The hardware structure of the device shown. Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 5As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0117] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0118] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0119] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0120] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0121] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0122] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0125] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A multi-label data instance cleaning method applied to the Internet of Things, characterized in that: The method is applied to IoT devices, including: Obtaining a target data instance to be cleaned; each target data instance is labeled with at least two candidate labels; The candidate labels of the target data instances are cleaned according to the candidate label contribution weights and specified principles by a trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels of each target data instance; the candidate label contribution weights are used to indicate the contribution of the candidate labels of the target data instances in determining the target label corresponding to the target data instance; The trained target data cleaning model has linear separable parameters, which are obtained by mapping training data instances into a high-dimensional space. The linear separable parameters are related to the contribution weights of the candidate labels labeled with each training data instance. The linear separable parameters are used to achieve linear separability of different target data instances input into the target data cleaning model. Among them, the designation principle is used to characterize the similarity of labels between data instances of the same type and the difference of labels between data instances of different types; the designation principle is: different target data instances marked with completely different candidate labels correspond to different target labels, and different target data instances marked with at least one identical candidate label and meeting similar conditions correspond to the same target label.

2. The method according to claim 1, characterized in that The different target data instances that meet similar conditions include: For the i-th target data instance x i , determine the i-th target data instance x i The kth nearest neighbor x ik ; According to the kth nearest neighbor x ik , and according to the semantic relevance between the i-th target data instance and the j-th target data instance, determine the similarity S between the i-th target data instance and the j-th target data instance ij ; When S ij is greater than or equal to the preset threshold, then it is determined that the i-th target data instance and the j-th target data instance are similar. ij If the value is less than a preset threshold, it is determined that the i-th target data instance and the j-th target data instance are not similar.

3. The method according to claim 1, characterized in that The target data cleaning model is trained in the following way: Obtaining a contribution weight of each candidate label for a training data instance in a training set; wherein the training data instance in the training set is labeled with at least one candidate label, wherein, when a training data instance is labeled with two or more candidate labels, one of the candidate labels is a target label for the training data instance; the contribution weight of the candidate label is used to indicate the contribution of the candidate label in determining the target label corresponding to the training data instance; The target data cleaning model is trained according to preset principles and the contribution weights of the candidate labels marked on each training data instance; the preset principles require that: training data instances marked with completely different candidate labels correspond to different target labels, and training data instances marked with at least one identical candidate label and meeting similar conditions correspond to the same target label.

4. The method according to claim 3, characterized in that The linear separability parameter is expressed by the following formula: = ； Wherein, W represents the linear separable parameter, Represents instances obtained by mapping training data instances to high-dimensional space Represents the configured Gaussian kernel function, and A represents the contribution weight of each candidate label for each training data instance.

5. The method according to claim 3, characterized in that The step of obtaining the contribution weight of each candidate label of the training data instance in the training set includes: Obtain the contribution weight of each candidate label for the training data instances in the pre-configured training set; or, Obtain the result obtained by optimizing the contribution weights of each candidate label of the training data instances in the pre-configured training set according to the specified optimization algorithm.

6. The method according to claim 3, characterized in that Training the target data cleaning model according to the preset principles and the contribution weights of the candidate labels of each training data instance includes: A reference data cleaning model is trained according to preset principles and the contribution weights of each candidate label of each training data instance; The reference data cleaning model is tested by using a test data instance in a test set, and when the reference data cleaning model meets a preset test condition, the reference data cleaning model is determined as the target data cleaning model.

7. The method according to claim 6, characterized in that The testing of the reference data cleaning model using the test data instance in the test set includes: Calculating the Gaussian function value between each test data instance and each training data instance by referring to the data cleaning model to obtain a Gaussian function value matrix; Gaussian function values ​​on the same row of the Gaussian function value matrix correspond to the same test data instance, and Gaussian function values ​​on different rows correspond to different test data instances; A prediction label matrix is ​​obtained by performing a specified operation on the Gaussian function value matrix and the obtained optimization weight matrix using a reference data cleaning model; the optimization weight matrix is ​​obtained by optimizing the contribution weights of each candidate label labeled with the training data instances in the training set according to a specified optimization algorithm; the prediction label matrix includes the predicted labels corresponding to the test data instances; Whether the reference data cleaning model meets the preset test conditions is determined based on the comparison result of the predicted label of the test data instance in the predicted label matrix and the real label configured for the test data instance; wherein, when the predicted label of the test data instance is similar to the real label configured for the test data instance, the comparison result for the test data instance is a match; otherwise, the comparison result is a mismatch.

8. The method according to any one of claims 1 to 7, characterized in that: The IoT device is an edge device deployed as an "edge" in the IoT, or, The IoT device is a terminal device deployed as an "end" in the IoT.

9. A multi-label data instance cleaning device applied to the Internet of Things, characterized in that: The device is used in IoT devices, including: An obtaining unit, configured to obtain a target data instance to be cleaned; each target data instance is labeled with at least two candidate labels; a cleaning unit, configured to clean the candidate labels labeled with the target data instances according to the candidate label contribution weights and a specified principle using a trained target data cleaning model based on partial labeling learning, so as to select a target label corresponding to each target data instance from at least two candidate labels labeled with each target data instance; the candidate label contribution weights are used to indicate the contribution of the candidate labels labeled with the target data instances in determining the target label corresponding to the target data instance; The trained target data cleaning model has linear separable parameters, which are obtained by mapping training data instances into a high-dimensional space. The linear separable parameters are related to the contribution weights of the candidate labels labeled with each training data instance. The linear separable parameters are used to achieve linear separability of different target data instances input into the target data cleaning model. Among them, the designation principle is used to characterize the similarity of labels between data instances of the same type and the difference of labels between data instances of different types; the designation principle is: different target data instances marked with completely different candidate labels correspond to different target labels, and different target data instances marked with at least one identical candidate label and meeting similar conditions correspond to the same target label.

10. The device according to claim 9, characterized in that The target data cleaning model is trained in the following way: Obtaining a contribution weight of each candidate label for a training data instance in a training set; wherein the training data instance in the training set is labeled with at least one candidate label, wherein, when a training data instance is labeled with two or more candidate labels, one of the candidate labels is a target label for the training data instance; the contribution weight of the candidate label is used to indicate the contribution of the candidate label in determining the target label corresponding to the training data instance; The target data cleaning model is trained according to preset principles and the contribution weights of the candidate labels marked on each training data instance; the preset principles require that: training data instances marked with completely different candidate labels correspond to different target labels, and training data instances marked with at least one identical candidate label and meeting similar conditions correspond to the same target label.

Images