Zero-shot learning method, device and equipment for training internet of things model and medium

Abstract

The application relates to the technical field of artificial intelligence and Internet of Things, and provides an Internet of Things model training method, device and equipment and medium based on zero sample learning, which can establish a feature vector space system comprising multi-dimensional features such as device attribute features, core data features and auxiliary features, and further improve the features in combination with artificial labeling and machine learning automatic labeling, utilize a preset convolutional neural network to establish a unified Internet of Things scene recognition model to assist in scene recognition of to-be-processed data, utilize distance based on a semantic space data feature vector to verify relevance, further check a scene recognition result, and when the verification is passed, train an Internet of Things model according to the recognized scene data, thereby solving the modeling problem of an Internet of Things scene lacking samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and Internet of Things, and in particular to an Internet of Things model training method and device based on zero-shot learning, equipment and medium. BACKGROUND

[0002] Generally, traditional machine learning modeling requires a large number of labeled samples, but it is difficult and costly to obtain standardized samples in reality, and after obtaining the samples, a large number of manually labeled samples are also required for training to obtain higher prediction accuracy according to the model learning task. Generally, each object class requires thousands of labeled samples, which is high in labeling cost and time-consuming.

[0003] Especially in the context of the Internet of Everything, a large number of sensors and terminal devices are connected to the network, providing a large amount of analyzable data objects for artificial intelligence, giving birth to a large number of new Internet of Things application scenarios, and the constantly emerging new businesses, new scenarios, and new formats do not necessarily have historical sample data open and accumulate.

[0004] Therefore, due to the problems of difficult data acquisition in some scenarios, extremely unbalanced positive and negative sample data, no historical sample data in new scenarios, or insufficient manual labeling, effective modeling cannot be performed in the corresponding Internet of Things scenarios. SUMMARY

[0005] In view of the above, it is necessary to provide an Internet of Things model training method and device based on zero-shot learning, and equipment and medium, aiming to solve the modeling problem in the Internet of Things scenario when lacking samples.

[0006] An Internet of Things model training method based on zero-shot learning, the Internet of Things model training method based on zero-shot learning comprises:

[0007] Obtain training samples, and obtain device attribute features, core data features, and auxiliary features of each training sample in the training samples;

[0008] Construct a first feature vector space system according to the device attribute features, core data features, and auxiliary features of each training sample;

[0009] Label the auxiliary information of each training sample, and map the labeled auxiliary information to the first feature vector space system to obtain a second feature vector space system;

[0010] Obtain the Internet of Things scenario category corresponding to each training sample, and establish the mapping relationship between the Internet of Things scenario category corresponding to each training sample and each feature vector in the second feature vector space system to obtain a feature vector association system;

[0011] The preset convolutional neural network is trained by using the feature vector association system, so as to obtain an Internet of Things scene identification model;

[0012] In response to an Internet of Things scene identification request based on to-be-processed data, the to-be-processed data is input into the Internet of Things scene identification model, and a predicted Internet of Things scene corresponding to the to-be-processed data is determined according to output data of the Internet of Things scene identification model;

[0013] The to-be-processed data is subjected to association verification;

[0014] When the to-be-processed data passes the association verification, the predicted Internet of Things scene is determined as a target Internet of Things scene corresponding to the to-be-processed data;

[0015] According to the target Internet of Things scene, data corresponding to the target Internet of Things scene is obtained from the feature vector association system, and a preset Internet of Things model corresponding to the to-be-processed data is trained.

[0016] According to the preferred embodiment of the present application, the device attribute feature is of a Boolean type or a real number type, and the device attribute feature includes one or a combination of the following: a device type, a function attribute, a performance attribute, and an industry attribute.

[0017] The core data feature is a quality feature of multi-domain heterogeneous data itself.

[0018] The auxiliary feature includes a device application scene description and a keyword feature.

[0019] According to the preferred embodiment of the present application, the construction of the first feature vector space system according to the device attribute feature, the core data feature, and the auxiliary feature of each training sample includes:

[0020] The device attribute feature, the core data feature, and the auxiliary feature of each training sample are encoded to obtain a vector representation of each training sample.

[0021] The first feature vector space system is constructed according to the vector representation of each training sample.

[0022] According to the preferred embodiment of the present application, the labeling of the auxiliary information of each training sample includes:

[0023] The attribute feature, the text keyword feature, and the syntax feature marked by a user are obtained as first labeling features.

[0024] A pre-constructed feature extraction model is used to extract a label vector space, a text vector space, and an image vector space of each training sample as second labeling features.

[0025] The first labeling features and the second labeling features are combined to obtain target labeling features.

[0026] According to the target annotation feature, the auxiliary information of each training sample is annotated.

[0027] According to the preferred embodiment of the present application, when the mapping relationship between each training sample and each feature vector in the second feature vector space system is established, the method further comprises:

[0028] A scene identifier is added to the corresponding Internet of Things scene category of each mapping relationship.

[0029] According to the preferred embodiment of the present application, the predicted Internet of Things scene comprises a predicted scene classification result; and the scene classification result is a multi-level classification.

[0030] According to the preferred embodiment of the present application, the correlation verification according to the to-be-processed data comprises:

[0031] The correlation between the to-be-processed data and the predicted Internet of Things scene is calculated as a first correlation;

[0032] The correlation between the training sample and the predicted Internet of Things scene is calculated as a second correlation;

[0033] The quotient of the first correlation and the second correlation is calculated as a target semantic similarity;

[0034] A preconfigured similarity threshold is obtained;

[0035] When the target semantic similarity is greater than or equal to the similarity threshold, it is determined that the to-be-processed data passes the correlation verification; or

[0036] When the target semantic similarity is less than the similarity threshold, it is determined that the to-be-processed data does not pass the correlation verification.

[0037] A zero-shot learning Internet of Things model training device, comprising:

[0038] An acquisition unit is configured to acquire training samples, and acquire device attribute features, core data features, and auxiliary features of each training sample in the training samples.

[0039] A construction unit is configured to construct a first feature vector space system according to the device attribute features, the core data features, and the auxiliary features of each training sample.

[0040] An annotation unit is configured to annotate auxiliary information of each training sample, and map the annotated auxiliary information to the first feature vector space system to obtain a second feature vector space system.

[0041] a mapping unit configured to obtain an Internet of Things (IoT) scene category corresponding to each training sample, and establish a mapping relationship between the IoT scene category corresponding to each training sample and each feature vector in the second feature vector space system, to obtain a feature vector association system;

[0042] a training unit configured to train a preset convolutional neural network using the feature vector association system, to obtain an IoT scene identification model;

[0043] an identification unit configured to, in response to an IoT scene identification request based on to-be-processed data, input the to-be-processed data into the IoT scene identification model, and determine a predicted IoT scene corresponding to the to-be-processed data according to output data of the IoT scene identification model;

[0044] a verification unit configured to perform association verification according to the to-be-processed data;

[0045] a determination unit configured to, when the to-be-processed data passes the association verification, determine that the predicted IoT scene is a target IoT scene corresponding to the to-be-processed data;

[0046] The training unit is further configured to train a preset IoT model corresponding to the to-be-processed data according to data obtained from the feature vector association system based on the target IoT scene.

[0047] A computer device, comprising:

[0048] a memory configured to store at least one instruction; and

[0049] a processor configured to execute the instruction stored in the memory to implement the zero-shot learning IoT model training method.

[0050] A computer-readable storage medium having at least one instruction stored therein, the at least one instruction being executed by a processor in a computer device to implement the zero-shot learning IoT model training method.

[0051] As can be seen from the above technical solutions, the present application can establish a feature vector space system including device attribute features, core data features, auxiliary features and the like, and further improve the features by combining artificial labeling and machine learning automatic labeling, and use a preset convolutional neural network to establish a unified IoT scene identification model to assist in scene identification of to-be-processed data. Further, the data feature vectors based on the semantic space are associated by distance for association verification, to further check the scene identification result. When the verification is passed, data corresponding to the identified scene is used to train an IoT model, thereby solving the problem of modeling of IoT scenes lacking samples. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a flow chart of a preferred embodiment of the IoT model training method of the present application for zero-shot learning.

[0053] Figure 2 is a functional module diagram of a preferred embodiment of the IoT model training device of the present application for zero-shot learning.

[0054] Figure 3 is a structural schematic diagram of a computer device of a preferred embodiment of the IoT model training method of the present application for zero-shot learning. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in detail below in combination with the drawings and specific embodiments.

[0056] As shown in Figure 1 is a flow chart of a preferred embodiment of the IoT model training method of the present application for zero-shot learning. The order of steps in the flow chart can be changed according to different needs, and some steps can be omitted.

[0057] The IoT model training method for zero-shot learning is applied in one or more computer devices, and the computer device is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, the hardware of which includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0058] The computer device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc.

[0059] The computer device can also include a network device and / or a user device. The network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

[0060] The server can be a standalone server or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0061] Artificial intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain optimal results.

[0062] Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, and the like. Artificial intelligence software technologies mainly include computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, and the like.

[0063] The network in which the computer device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), and the like.

[0064] S10, obtaining a training sample, and obtaining a device attribute feature, a core data feature, and an auxiliary feature of each training sample in the training sample.

[0065] In this embodiment, the training sample can be data in an Internet of Things scenario where samples are easily obtained and sample quantities are sufficient.

[0066] In this embodiment, the device attribute feature is of a Boolean type or a real number type; the device attribute feature can include, but is not limited to, one or a combination of the following: a device type, a functional attribute, a performance attribute, and an industry attribute.

[0067] The device type can include a camera, a sensor, and the like.

[0068] In this embodiment, the core data feature is a quality feature of multi-domain heterogeneous data itself.

[0069] For example, the core data feature can include heterogeneous data descriptions such as video, image, and voice, such as a sampling rate, a format (such as PCM / wav), a length, and the like of audio, a coding and decoding format, a length, and the like of video, and a picture format size.

[0070] In the embodiment, the auxiliary feature can include device application scene description, keyword feature, etc.

[0071] S11, constructing a first feature vector space system according to the device attribute feature, the core data feature and the auxiliary feature of each training sample.

[0072] In the embodiment, the constructing a first feature vector space system according to the device attribute feature, the core data feature and the auxiliary feature of each training sample includes:

[0073] Encoding the device attribute feature, the core data feature and the auxiliary feature of each training sample to obtain a vector representation of each training sample;

[0074] Constructing the first feature vector space system according to the vector representation of each training sample.

[0075] For example, after encoding, φ(x) represents the device attribute feature, θ(x) represents the core data feature, and w represents the auxiliary feature. In the first feature vector space system, (θ(x), φ(x); w) represents a feature vector.

[0076] In the above embodiment, a multi-dimensional and multi-domain sample feature space is constructed. Compared with using a single attribute (such as flow analysis or image analysis) as a training sample, more dimensional feature data such as device attribute feature, core data feature and auxiliary feature is added, so that the finally obtained feature vector space system is more comprehensive.

[0077] S12, labeling the auxiliary information of each training sample, and mapping the labeled auxiliary information to the first feature vector space system to obtain a second feature vector space system.

[0078] In the embodiment, the labeling the auxiliary information of each training sample includes:

[0079] Obtaining the attribute feature, the text keyword feature and the syntax feature marked by the user as first labeling features;

[0080] Extracting the label vector space, the text vector space and the image vector space of each training sample as second labeling features by using a pre-constructed feature extraction model;

[0081] Combining the first labeling features and the second labeling features to obtain target labeling features;

[0082] Labeling the auxiliary information of each training sample according to the target labeling features.

[0083] The feature extraction model can be a Bag-of-Word, Word2Vec or other machine learning model.

[0084] Further map the labeled auxiliary information to the first feature vector space system to obtain the second feature vector space system.

[0085] In the above embodiment, feature extraction is further combined with feature extraction model based on manual labeling to increase feature values of auxiliary information, improve accuracy of auxiliary standard information, reduce uncertainty, and facilitate recognition and application of new scenarios.

[0086] S13, obtain the Internet of Things scene category corresponding to each training sample, and establish a mapping relationship between the Internet of Things scene category corresponding to each training sample and each feature vector in the second feature vector space system to obtain a feature vector association system.

[0087] For example, in the feature vector association system, the mapping relationship can be represented as: f (θ (x), φ (x) ; w) → Y, wherein Y is the Internet of Things scene category.

[0088] In this embodiment, the Internet of Things scene category corresponding to each training sample is predetermined.

[0089] In this embodiment, when establishing the mapping relationship between the Internet of Things scene category corresponding to each training sample and each feature vector in the second feature vector space system, the method further comprises:

[0090] Adding a scene identifier to the corresponding Internet of Things scene category for each mapping relationship.

[0091] The scene identifier is used to uniquely mark the corresponding Internet of Things scene.

[0092] S14, training a preset convolutional neural network using the feature vector association system to obtain an Internet of Things scene recognition model.

[0093] Specifically, in the training process, each feature vector in the feature vector association system is used as input data, the corresponding Internet of Things scene is used as a training target, and the training is stopped when the model converges to obtain the Internet of Things scene recognition model.

[0094] Specifically, in the training process, the weight of the new scene sample data and the auxiliary information description is adjusted using the parameter sharing capability of the training stage convolutional layer, and the model is provided with basic detection function for zero sample data through multi-stage training.

[0095] In the above embodiment, the extracted feature space is uniformly processed by using a convolutional neural network model, including the device description feature attribute, the core data feature, the scene classification space, and the connection between the embedded auxiliary knowledge factors, so as to facilitate understanding, training and use, and solve the problem of weak generalization ability of a specific linear or nonlinear function expression.

[0096] S15, in response to an Internet of Things scene identification request based on the to-be-processed data, inputting the to-be-processed data into the Internet of Things scene identification model, and determining a predicted Internet of Things scene corresponding to the to-be-processed data according to output data of the Internet of Things scene identification model.

[0097] In the embodiment, the to-be-processed data belongs to data in an Internet of Things scene with a small amount of data.

[0098] Further, a query vector is formed according to the to-be-processed data.

[0099] Specifically, the input data features are analyzed, and a query vector is given in combination with scene expression; meanwhile, auxiliary description shared with the training sample is adopted, so as to obtain the identification / classification result of the unknown category data through the mapping relationship.

[0100] In the embodiment, the predicted Internet of Things scene includes a predicted scene classification result; and the scene classification result is multi-level classification. For example, the scene classification result can include a main class, a sub-class, etc.

[0101] In the embodiment, scene subdivision or multi-level classification can be performed according to the granularity of the sample data feature extraction.

[0102] In the embodiment, the output data can include a scene identifier of the predicted Internet of Things scene, and the predicted Internet of Things scene can be located through the scene identifier.

[0103] S16, performing correlation verification according to the to-be-processed data.

[0104] In the embodiment, the correlation verification according to the to-be-processed data includes:

[0105] Calculating the correlation between the to-be-processed data and the predicted Internet of Things scene as a first correlation;

[0106] Calculating the correlation between the training sample and the predicted Internet of Things scene as a second correlation;

[0107] Calculating a quotient of the first correlation and the second correlation as a target semantic similarity;

[0108] Obtaining a pre-configured similarity threshold;

[0109] When the target semantic similarity is greater than or equal to the similarity threshold, it is determined that the to-be-processed data passes the correlation verification; or

[0110] When the target semantic similarity is less than the similarity threshold, it is determined that the to-be-processed data does not pass the correlation verification.

[0111] The similarity threshold can be configured according to a large number of experiments.

[0112] For example, the target semantic similarity can be calculated by using the following formula:

[0113]

[0114] Wherein, d represents the target semantic similarity, L represents a smoothing coefficient of semantic labeling, k represents a feature correlation weight coefficient; i, t represent the number of data features, w represents a basic data feature metric distance based on semantic feature dimension extraction, parameter φ represents a semantic labeling feature description mapping difference distance based on dimension; parameter θ represents a device terminal attribute description feature distance based on semantics.

[0115] Specifically, the semantic space feature distance metric can generally be calculated by using Hamming distance or cosine distance.

[0116] In the above embodiment, the semantic space-based data feature vector is used to calculate the distance to analyze the correlation degree or similarity between the unknown set and the classification label, so as to obtain the recognition / classification further evaluation of the zero-sample unknown class data.

[0117] S17, when the to-be-processed data passes the correlation verification, it is determined that the predicted Internet of Things scene is a target Internet of Things scene corresponding to the to-be-processed data.

[0118] In this embodiment, when the to-be-processed data does not pass the correlation verification, a prompt information can be sent to re-perform scene identification.

[0119] S18, according to the target Internet of Things scene, a data pair corresponding to the to-be-processed data is obtained from the feature vector correlation system to train a preset Internet of Things model.

[0120] For example, there are problems such as few open data samples, difficult acquisition of high-quality samples, and high cost of manual labeling in the application of Internet of Things and industrial intelligence. Therefore, the present embodiment can be applied to fast learning modeling in the scene of Internet of Things and industrial data analysis under small sample or even zero sample, effectively solving the problem of unbalanced positive and negative samples in the scene of industrial intelligence, assisting the rapid migration and identification matching of new scenes in the application of Internet of Things or industry, and being beneficial to the identification matching of new scenes in Internet of Things and industry.

[0121] Through the above embodiments, the IoT scenario corresponding to the data to be processed is first determined by scene recognition, and then the corresponding IoT model is trained using the data corresponding to the scenario. Sample data of new scenario applications are obtained based on the concept of small sample and zero sample learning. New scenario samples generally have characteristics such as small quantity or typicality. Then, by using semantic description auxiliary information of unseen class samples and comprehensively utilizing the attribute features of the training samples in the overall dataset, the new category is learned and modeled.

[0122] As can be seen from the above technical solutions, the present invention can establish a feature vector space system with multi-dimensional features including device attribute features, core data features, and auxiliary features. It further improves the features by combining manual annotation and automatic annotation by machine learning. It uses a pre-set convolutional neural network to establish a unified IoT scene recognition model to assist in scene recognition of the data to be processed. Furthermore, it uses distance to verify the correlation of data feature vectors based on semantic space to further verify the scene recognition results. Once the verification is passed, data can be obtained from the recognized scene to train the IoT model, thus solving the problem of modeling IoT scenes with a lack of samples.

[0123] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the zero-shot learning IoT model training device of the present invention. The zero-shot learning IoT model training device 11 includes an acquisition unit 110, a construction unit 111, a labeling unit 112, a mapping unit 113, a training unit 114, an identification unit 115, a verification unit 116, and a determination unit 117. The module / unit referred to in this invention refers to a series of computer program segments that can be executed by a processor and perform a fixed function, and are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0124] The acquisition unit 110 is used to acquire training samples, and to acquire the device attribute features, core data features, and auxiliary features of each training sample in the training samples.

[0125] In this embodiment, the training samples can be data from IoT scenarios where samples are readily available and the sample size is sufficient.

[0126] In this embodiment, the device attribute features are Boolean or real number types; the device attribute features may include, but are not limited to, one or more of the following combinations: device type, functional attributes, performance attributes, and industry attributes;

[0127] The device types mentioned may include cameras, sensors, etc.

[0128] In the embodiment, the core data features are the quality features of the multi-domain heterogeneous data itself.

[0129] For example, the core data features can include heterogeneous data descriptions such as video, image, voice, etc., such as the sampling rate, format (such as PCM / wav), length, etc. of the audio, the coding format, length, etc. of the video, and the picture format size, etc.

[0130] In the embodiment, the auxiliary features can include device application scenario descriptions, keyword features, etc.

[0131] The construction unit 111 is configured to construct a first feature vector space system according to the device attribute features, the core data features, and the auxiliary features of each training sample.

[0132] In the embodiment, the construction unit 111 constructs the first feature vector space system according to the device attribute features, the core data features, and the auxiliary features of each training sample, and the construction includes:

[0133] Encoding the device attribute features, the core data features, and the auxiliary features of each training sample to obtain a vector representation of each training sample.

[0134] The first feature vector space system is constructed according to the vector representation of each training sample.

[0135] For example, after encoding, φ(x) represents the device attribute features, θ(x) represents the core data features, and w represents the auxiliary features. In the first feature vector space system, (θ(x), φ(x); w) represents a feature vector.

[0136] In the above embodiment, a multi-dimensional and multi-domain sample feature space is constructed. Compared with using a single attribute (such as flow analysis or image analysis) as a training sample, more dimensional feature data such as device attribute features, core data features, and auxiliary features are added, so that the finally obtained feature vector space system is more comprehensive.

[0137] The labeling unit 112 is configured to label the auxiliary information of each training sample and map the labeled auxiliary information to the first feature vector space system to obtain a second feature vector space system.

[0138] In the embodiment, the labeling unit 112 labels the auxiliary information of each training sample, and the labeling includes:

[0139] Obtaining the attribute features, the text keyword features, and the syntax features marked by the user as first labeling features.

[0140] extracting a label vector space, a text vector space and an image vector space of each training sample as second annotation features by using a pre-constructed feature extraction model;

[0141] combining the first annotation features and the second annotation features to obtain target annotation features;

[0142] annotating auxiliary information of each training sample according to the target annotation features.

[0143] The feature extraction model can be a Bag-of-Word, a Word2Vec or other machine learning model.

[0144] further mapping the annotated auxiliary information to the first feature vector space system to obtain the second feature vector space system.

[0145] In the above embodiment, feature extraction is further performed on the basis of manual annotation by combining a feature extraction model, so as to increase the feature values of auxiliary information, improve the accuracy of standard auxiliary information, reduce uncertainty, and facilitate the recognition and application of new scenarios.

[0146] The mapping unit 113 is configured to obtain an Internet of Things scenario category corresponding to each training sample, establish a mapping relationship between the Internet of Things scenario category corresponding to each training sample and each feature vector in the second feature vector space system, and obtain a feature vector association system.

[0147] For example, in the feature vector association system, the mapping relationship can be represented as f (θ (x), φ (x) ; w) → Y, where Y is an Internet of Things scenario category.

[0148] In this embodiment, the Internet of Things scenario category corresponding to each training sample is pre-determined.

[0149] In this embodiment, when the mapping relationship between the Internet of Things scenario category corresponding to each training sample and each feature vector in the second feature vector space system is established, a scenario identifier is added to the corresponding Internet of Things scenario category for each mapping relationship.

[0150] The scenario identifier is used to uniquely mark the corresponding Internet of Things scenario.

[0151] The training unit 114 is configured to train a pre-set convolutional neural network by using the feature vector association system to obtain an Internet of Things scenario recognition model.

[0152] Specifically, in the training process, each feature vector in the feature vector association system is used as input data, the corresponding Internet of Things scenario is used as a training target, and the training is stopped when the model converges, so as to obtain the Internet of Things scenario recognition model.

[0153] Specifically, in the training process, the weight of the new scene sample data and the auxiliary information description is adjusted by using the parameter sharing capability of the training stage convolution layer, and the model is trained in multiple stages to have a basic detection function for zero sample data.

[0154] In the above embodiment, the extracted feature space is uniformly processed by using a convolutional neural network model, including the connection between the device description feature attribute, the core data feature, the scene classification space, and the embedded auxiliary knowledge factor, which is convenient for understanding, training and use, and solves the problem of weak generalization ability of a specific linear or nonlinear function expression.

[0155] The recognition unit 115 is configured to input the to-be-processed data into the Internet of Things scene recognition model in response to an Internet of Things scene recognition request based on the to-be-processed data, and determine a predicted Internet of Things scene corresponding to the to-be-processed data according to output data of the Internet of Things scene recognition model.

[0156] In this embodiment, the to-be-processed data belongs to data in an Internet of Things scene with a small amount of data.

[0157] Further, a query vector is formed according to the to-be-processed data.

[0158] Specifically, the input data characteristics are analyzed, and a query vector is given in combination with scene expression; meanwhile, auxiliary description is shared with the training sample, so as to obtain the recognition / classification result of unknown category data through the mapping relationship.

[0159] In this embodiment, the predicted Internet of Things scene includes a predicted scene classification result; and the scene classification result is a multi-level classification. For example, the scene classification result can include a main class, a sub-class, and the like.

[0160] In this embodiment, scene subdivision or multi-level classification can be performed according to the granularity of sample data feature extraction.

[0161] In this embodiment, the scene identifier of the predicted Internet of Things scene can be included in the output data, and the predicted Internet of Things scene can be located through the scene identifier.

[0162] The verification unit 116 is configured to perform associativity verification according to the to-be-processed data.

[0163] In this embodiment, the verification unit 116 performs associativity verification according to the to-be-processed data, which includes:

[0164] The associativity between the to-be-processed data and the predicted Internet of Things scene is calculated as a first associativity;

[0165] The associativity between the training sample and the predicted Internet of Things scene is calculated as a second associativity;

[0166] calculating a quotient of the first correlation and the second correlation as a target semantic similarity;

[0167] obtaining a pre-configured similarity threshold;

[0168] when the target semantic similarity is greater than or equal to the similarity threshold, determining that the to-be-processed data passes the correlation verification; or

[0169] when the target semantic similarity is less than the similarity threshold, determining that the to-be-processed data fails the correlation verification.

[0170] The similarity threshold can be configured according to a large number of experiments.

[0171] For example, the target semantic similarity can be calculated by using the following formula:

[0172]

[0173] wherein d represents the target semantic similarity, L represents a smoothing coefficient of semantic labeling, k represents a feature correlation weight coefficient; i and t represent the number of data features, w represents a basic data feature metric distance extracted based on a semantic feature dimension, parameter φ represents a semantic labeling feature description mapping difference distance based on a dimension; and parameter θ represents a device terminal attribute description feature distance based on semantics.

[0174] Specifically, the semantic space feature distance metric can be calculated by using a Hamming distance or a cosine distance.

[0175] In the above embodiment, the distance calculation is used to analyze the correlation or similarity between the unknown set and the classification label based on the semantic space data feature vector, so as to obtain the further evaluation of the recognition / classification of the zero-sample unknown class data.

[0176] The determination unit 117 is configured to determine that the predicted Internet of Things scene is a target Internet of Things scene corresponding to the to-be-processed data when the to-be-processed data passes the correlation verification.

[0177] In this embodiment, when the to-be-processed data fails the correlation verification, a prompt information can be sent to re-perform scene identification.

[0178] The training unit 114 is further configured to obtain a preset Internet of Things model corresponding to the to-be-processed data from the feature vector correlation system according to the target Internet of Things scene, and train the preset Internet of Things model.

[0179] For example: for the Internet of Things, industrial intelligence and other applications, there are generally few open data samples, it is difficult to obtain high-quality samples, and the cost of manual annotation is high. Therefore, the embodiment can be applied to fast learning modeling in the Internet of Things and industrial data analysis scenarios under small sample or even zero sample, effectively solving the problem of unbalanced positive and negative samples in the industrial intelligence scenario, assisting the rapid migration and identification matching of new scenarios in the Internet of Things or industrial applications, and facilitating the identification and matching of new scenarios in the Internet of Things and industry.

[0180] Through the above embodiment, the scene recognition is first determined according to the to-be-processed data corresponding to the Internet of Things scene, and then the corresponding Internet of Things model is trained by using the data corresponding to the scene. The sample data of the new scene application is obtained based on the concept of small sample and zero sample learning. The new scene sample generally has the characteristics of small amount or typicality, and then the attribute feature overall data set of the training sample is comprehensively utilized based on the semantic description auxiliary information of the unobserved class sample to learn and model the new class.

[0181] As can be seen from the above technical solutions, the present application can establish a feature vector space system including device attribute features, core data features, auxiliary features and the like, and further improve the features by combining artificial annotation and machine learning automatic annotation. A unified Internet of Things scene identification model is established by using a preset convolutional neural network to assist in scene identification of to-be-processed data. Further, the distance is used for relevance verification based on the data feature vector of the semantic space to further check the scene identification result. When the verification is passed, the data of the identified scene can be used to train the Internet of Things model, thereby solving the problem of modeling the Internet of Things scene lacking samples.

[0182] As shown in FIG. 1, it is a structural schematic diagram of a computer device of a preferred embodiment of the Internet of Things model training method for zero sample learning. Figure 3

[0183] The computer device 1 can include a memory 12, a processor 13 and a bus, and can also include a computer program stored in the memory 12 and executable on the processor 13, such as an Internet of Things model training program for zero sample learning.

[0184] Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1 and does not constitute a limitation on the computer device 1. The computer device 1 can be a bus type structure or a star type structure. The computer device 1 can also include more or less other hardware or software, or different component arrangements, such as the computer device 1 can also include an input / output device, a network access device, etc.

[0185] ​It should be noted that the computer device 1 is only an example, and other existing or future electronic products can also be adapted to the present application and should be included in the protection scope of the present application.

[0186] The memory 12 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 12 can be an internal storage unit of the computer device 1 in some embodiments, such as a mobile hard disk of the computer device 1. The memory 12 can also be an external storage device of the computer device 1 in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 12 can include both an internal storage unit and an external storage device of the computer device 1. The memory 12 can be used to store application software and various data installed in the computer device 1, such as the code of the Internet of Things model training program for zero-shot learning, and can also be used to temporarily store data that has been output or will be output.

[0187] The processor 13 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 13 is the control unit of the computer device 1, which connects various components of the computer device 1 through various interfaces and lines, executes programs or modules stored in the memory 12 (such as the Internet of Things model training program for zero-shot learning, etc.), and calls data stored in the memory 12 to perform various functions and process data of the computer device 1.

[0188] The processor 13 executes the operating system and various application programs installed in the computer device 1. The processor 13 executes the application programs to implement the steps in each of the above-described Internet of Things model training methods for zero-shot learning, such as Figure 1 the steps shown in the above embodiments.

[0189] The computer program can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units can be a series of computer-readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the computer device 1. For example, the computer program can be divided into an acquisition unit 110, a construction unit 111, a labeling unit 112, a mapping unit 113, a training unit 114, an identification unit 115, a verification unit 116, and a determination unit 117.

[0190] The integrated units in the form of software function modules described above can be stored in a computer-readable storage medium. The software function modules described above are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute part of the zero-shot learning Internet of Things model training method described in various embodiments of the present application.

[0191] The modules / units integrated in the computer device 1, if realized in the form of software function units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiments of the method can also be instructed by a computer program to complete related hardware devices, and the computer program can be stored in a computer-readable storage medium. When the processor executes the computer program, the steps of each method embodiment described above can be implemented.

[0192] The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0193] Further, the computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; and the data storage area can store data created according to the use of the blockchain node, etc.

[0194] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block contains a batch of network transaction information, used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0195] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one straight line is used in Figure 3 , but it does not mean that there is only one bus or one type of bus. The bus is arranged to realize the connection and communication between the memory 12, the at least one processor 13, etc.

[0196] Although not shown, the computer device 1 can also include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, so that the power management device can realize functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The computer device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described here.

[0197] Further, the computer device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the computer device 1 and other computer devices.

[0198] Optionally, the computer device 1 can further include a user interface, which can be a display, an input unit such as a keyboard, and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, for displaying information processed in the computer device 1 and for displaying a visualized user interface.

[0199] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by the structure.

[0200] Figure 3 Only the computer device 1 with components 12-13 is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the computer device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0201] In combination Figure 1 The memory 12 in the computer device 1 stores a plurality of instructions to implement a zero-shot learning Internet of Things model training method, and the processor 13 can execute the plurality of instructions to implement:

[0202] Obtaining training samples, and obtaining device attribute features, core data features, and auxiliary features of each training sample in the training samples;

[0203] Constructing a first feature vector space system according to the device attribute features, the core data features, and the auxiliary features of each training sample;

[0204] Labeling the auxiliary information of each training sample, and mapping the labeled auxiliary information to the first feature vector space system to obtain a second feature vector space system;

[0205] Obtaining an Internet of Things scene category corresponding to each training sample, and establishing a mapping relationship between the Internet of Things scene category corresponding to each training sample and each feature vector in the second feature vector space system to obtain a feature vector association system;

[0206] Training a preset convolutional neural network using the feature vector association system to obtain an Internet of Things scene recognition model;

[0207] In response to a request for identifying an Internet of Things (IoT) scenario based on to-be-processed data, the to-be-processed data is input into the IoT scenario identification model, and a predicted IoT scenario corresponding to the to-be-processed data is determined according to output data of the IoT scenario identification model;

[0208] correlation verification is performed according to the to-be-processed data;

[0209] When the to-be-processed data passes the correlation verification, the predicted IoT scenario is determined as a target IoT scenario corresponding to the to-be-processed data.

[0210] According to the target IoT scenario, data is obtained from the feature vector correlation system, and a preset IoT model corresponding to the to-be-processed data is trained.

[0211] Specifically, the specific implementation method of the processor 13 to the above instructions can refer to Figure 1 The description of related steps in the corresponding embodiments will not be repeated here.

[0212] It should be noted that the data involved in the present case are all legally obtained.

[0213] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other means. For example, the above-described device embodiments are only schematic, for example, the division of the modules is only a logical function division, and other division manners can be used in actual implementation.

[0214] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0215] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, i.e., may be located in one place, or may be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0216] In addition, each functional module in various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0217] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0218] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0219] Furthermore, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the present application can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names and not to indicate any particular order.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A zero-shot learning method for training an Internet of Things (IoT) model, characterized in that, The zero-shot learning method for training IoT models includes: Acquire training samples, and acquire device attribute features, core data features, and auxiliary features for each training sample; wherein, the device attribute features are Boolean or real number types, and the device attribute features include one or more combinations of the following: device type, functional attributes, performance attributes, and industry attributes; the core data features are the quality features of the multi-domain heterogeneous data itself; the auxiliary features include device application scenario descriptions and keyword features; A first feature vector space system is constructed based on the device attribute features, core data features, and auxiliary features of each training sample; The auxiliary information of each training sample is labeled, and the labeled auxiliary information is mapped to the first feature vector space system to obtain the second feature vector space system; wherein, the labeling of the auxiliary information of each training sample includes: obtaining the attribute features, text keyword features, and grammatical features of user tags as first labeling features; using a pre-built feature extraction model to extract the label vector space, text vector space, and image vector space of each training sample as second labeling features; combining the first labeling features and the second labeling features to obtain target labeling features; and labeling the auxiliary information of each training sample according to the target labeling features; Obtain the IoT scene category corresponding to each training sample, and establish the mapping relationship between the IoT scene category corresponding to each training sample and each feature vector in the second feature vector space system to obtain the feature vector association system; A pre-defined convolutional neural network is trained using the aforementioned feature vector association system to obtain an IoT scene recognition model; In response to an IoT scene recognition request based on data to be processed, the data to be processed is input into the IoT scene recognition model, and the predicted IoT scene corresponding to the data to be processed is determined according to the output data of the IoT scene recognition model. The correlation verification based on the data to be processed includes: calculating the correlation between the data to be processed and the predicted IoT scenario as a first correlation; calculating the correlation between the training sample and the predicted IoT scenario as a second correlation; calculating the quotient of the first correlation and the second correlation as a target semantic similarity; obtaining a pre-configured similarity threshold; determining that the data to be processed passes the correlation verification when the target semantic similarity is greater than or equal to the similarity threshold; or determining that the data to be processed fails the correlation verification when the target semantic similarity is less than the similarity threshold. When the data to be processed passes the correlation verification, the predicted IoT scenario is determined to be the target IoT scenario corresponding to the data to be processed. Based on the target IoT scenario, data is obtained from the feature vector association system to train a preset IoT model corresponding to the data to be processed.

2. The zero-shot learning method for training IoT models as described in claim 1, characterized in that, The construction of the first feature vector space system based on the device attribute features, core data features, and auxiliary features of each training sample includes: The device attribute features, core data features, and auxiliary features of each training sample are encoded to obtain a vector representation of each training sample; The first feature vector space system is constructed based on the vector representation of each training sample.

3. The zero-shot learning method for training IoT models as described in claim 1, characterized in that, When establishing the mapping relationship between the IoT scene category corresponding to each training sample and each feature vector in the second feature vector space system, the method further includes: Add a scene identifier to the corresponding IoT scene category for each mapping relationship.

4. The zero-shot learning method for training IoT models as described in claim 1, characterized in that, The predicted IoT scenarios include predicted scenario classification results; the scenario classification results are multi-level classifications.

5. A zero-shot learning Internet of Things (IoT) model training device, characterized in that, The zero-shot learning IoT model training device includes: The acquisition unit is used to acquire training samples, and to acquire device attribute features, core data features, and auxiliary features for each training sample in the training samples; wherein, the device attribute features are Boolean or real number types, and the device attribute features include one or more combinations of the following: device type, functional attributes, performance attributes, and industry attributes; the core data features are the quality features of the multi-domain heterogeneous data itself; the auxiliary features include device application scenario descriptions and keyword features; The construction unit is used to construct the first feature vector space system based on the device attribute features, core data features, and auxiliary features of each training sample; A labeling unit is used to label the auxiliary information of each training sample and map the labeled auxiliary information to the first feature vector space system to obtain a second feature vector space system. The labeling of the auxiliary information of each training sample includes: obtaining user-marked attribute features, text keyword features, and grammatical features as first labeling features; using a pre-built feature extraction model to extract the label vector space, text vector space, and image vector space of each training sample as second labeling features; combining the first labeling features and the second labeling features to obtain target labeling features; and labeling the auxiliary information of each training sample according to the target labeling features. The mapping unit is used to obtain the IoT scene category corresponding to each training sample and establish the mapping relationship between the IoT scene category corresponding to each training sample and each feature vector in the second feature vector space system to obtain the feature vector association system. The training unit is used to train a preset convolutional neural network using the feature vector association system to obtain an Internet of Things scene recognition model; The identification unit is used to respond to an IoT scene identification request based on data to be processed, input the data to be processed into the IoT scene identification model, and determine the predicted IoT scene corresponding to the data to be processed based on the output data of the IoT scene identification model. A verification unit is configured to perform correlation verification based on the data to be processed, including: calculating the correlation between the data to be processed and the predicted IoT scenario as a first correlation; calculating the correlation between the training sample and the predicted IoT scenario as a second correlation; calculating the quotient of the first correlation and the second correlation as a target semantic similarity; obtaining a pre-configured similarity threshold; determining that the data to be processed passes the correlation verification when the target semantic similarity is greater than or equal to the similarity threshold; or determining that the data to be processed fails the correlation verification when the target semantic similarity is less than the similarity threshold. The determining unit is configured to determine the predicted IoT scenario as the target IoT scenario corresponding to the data to be processed when the data to be processed passes the correlation verification. The training unit is also used to obtain data from the feature vector association system based on the target IoT scenario to train a preset IoT model corresponding to the data to be processed.

6. A computer device, characterized in that, The computer device includes: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the zero-shot learning IoT model training method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the zero-shot learning Internet of Things model training method as described in any one of claims 1 to 4.

Images