Model selection method, device, equipment, medium and program product of passive internet-of-things label

Abstract

The invention provides a passive Internet of Things label type selection method and device, equipment, a medium and a program product, and relates to the technical field of computers.The method comprises the steps that a passive Internet of Things label type node serves as a center node, and a passive Internet of Things label topological graph composed of the passive Internet of Things label type node, a design scene node and a packaging material node is constructed; the node features in the passive Internet of Things tag topological graph are input into a passive Internet of Things tag topological feature extractor composed of GAT, and passive Internet of Things tag type new features are output; inputting the target scene image of the passive internet-of-things tag into a target scene image feature extractor formed by an LVM, and outputting a target scene image feature; and determining the type selection information of the passive Internet of Things label of the target scene according to the passive Internet of Things label type new feature and the target scene image feature. By combining the graph attention neural network and the large visual model, the model selection of the passive Internet of Things label can be accurately and efficiently realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment, medium and program product for selecting a passive Internet of Things tag. Background Art

[0002] With the deepening of industrial digitalization, the industry has an increasingly urgent need for intelligent, automated, and visualized management of all production factors and the entire life cycle, including personnel, assets, materials, and products. How to quickly select 5G-A (5G-Advanced, 5G enhanced version) passive IoT tags suitable for industry scenarios has become a challenging issue that needs to be solved. However, the selection of passive IoT tags is currently mainly based on the manual experience of experts, which is time-consuming, labor-intensive, inefficient, and inaccurate, thus affecting the promotion and application of 5G-A passive IoT tags. Therefore, how to accurately and efficiently select passive IoT tags has become an urgent problem to be solved. Summary of the invention

[0003] The present invention provides a method, device, equipment, medium and program product for selecting a passive Internet of Things tag, so as to solve the defects of how to accurately and efficiently realize the selection of passive Internet of Things tags in the prior art, and realize accurate and efficient selection of passive Internet of Things tags.

[0004] The present invention provides a method for selecting a passive IoT tag, comprising the following steps: Taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map consisting of the passive IoT tag type node, the design scenario node of the passive IoT tag, and the packaging material node of the passive IoT tag; Input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT, and obtain a new passive IoT tag type feature output by the passive IoT tag topology feature extractor; Inputting the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM, and obtaining the target scene image features output by the target scene image feature extractor; According to the new features of the passive Internet of Things tag type and the target scene image features, selection information of the passive Internet of Things tag of the target scene is determined.

[0005] According to a method for selecting a passive IoT tag provided by the present invention, the passive IoT tag topological feature extractor includes a word embedding layer, a neighbor node level graph attention layer and a center node level graph attention layer; The word embedding layer is used to convert the input node features into vector representations to obtain node feature vectors, and input the node feature vectors into the neighbor node level graph attention layer; The neighbor node level graph attention layer is used to perform neighbor node level attention aggregation on the node feature vector, and input the aggregated feature vector obtained by aggregation into the central node level graph attention layer; The central node-level graph attention layer is used to perform central node-level attention aggregation on the aggregated feature vector to obtain a new passive IoT tag type feature that simultaneously integrates features of multiple types of neighboring nodes; the neighboring nodes include the design scenario node and the packaging material node.

[0006] According to a method for selecting a passive IoT tag provided by the present invention, the word embedding layer includes a first word embedding layer, a second word embedding layer and a third word embedding layer, and the neighbor node level graph attention layer includes a first neighbor node level graph attention layer and a second neighbor node level graph attention layer; The first word embedding layer is used to convert the input passive Internet of Things tag type feature into a vector representation to obtain a passive Internet of Things tag type feature vector, and input the passive Internet of Things tag type feature vector into the first neighbor node level graph attention layer and the second neighbor node level graph attention layer; The second word embedding layer is used to convert the input design scene feature into a vector representation to obtain a design scene feature vector, and input the design scene feature vector into the first neighbor node level graph attention layer; The third word embedding layer is used to convert the input packaging material feature into a vector representation to obtain a packaging material feature vector, and input the packaging material feature vector into the second neighbor node level graph attention layer; The first neighbor node level graph attention layer is used to calculate the first attention weight of the design scenario node to the current central node according to the passive IoT tag type feature vector and the design scenario feature vector, and perform weighted aggregation on the features of the design scenario node according to the first attention weight to obtain a first aggregated feature vector; The second neighbor node level graph attention layer is used to calculate the second attention weight of the packaging material node to the current central node based on the passive IoT tag type feature vector and the packaging material feature vector, and perform weighted aggregation on the features of the neighbor nodes according to the attention weight to obtain a second aggregated feature vector.

[0007] According to a method for selecting a passive IoT tag provided by the present invention, the target scene image includes a target scene carrier morphology image and a target scene application environment image; the target scene image features include a target carrier morphology image feature vector and a target scene environment image feature vector; the LVM includes a fully connected layer and a pooling layer.

[0008] According to a method for selecting a passive IoT tag provided by the present invention, the method of determining the selection information of the passive IoT tag of the target scene according to the new feature of the passive IoT tag type and the feature of the target scene image includes: The new passive IoT tag type feature and the target scene image feature are input into a passive IoT tag type matcher to obtain the selection information of the passive IoT tag of the target scene output by the passive IoT tag type matcher.

[0009] According to a passive IoT tag selection method provided by the present invention, the passive IoT tag type matcher includes a merging layer, a fully connected attention layer and an output layer; The merging layer is used to merge the new feature of the passive IoT tag type and the target scene image feature into feature vectors, and input the merged feature vectors into the fully connected attention layer; The fully connected attention layer is used to determine the attention weight of each region in the target scene image according to the merged feature vector, and weight the image information of the target scene image based on the attention weight of each region to obtain adjusted image information; The output layer is used to output the selection information of the passive IoT tag of the target scene based on the adjusted image information.

[0010] The present invention also provides a device for selecting a passive IoT tag, comprising the following modules: A construction module is used to construct a passive IoT tag topology map consisting of a passive IoT tag type node, a design scenario node of a passive IoT tag, and a packaging material node of the passive IoT tag, with the passive IoT tag type node as a central node; A first feature extraction module is used to input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT to obtain a new feature of the passive IoT tag type output by the passive IoT tag topology feature extractor; A second feature extraction module is used to input the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM to obtain the target scene image features output by the target scene image feature extractor; A selection module is used to determine the selection information of the passive Internet of Things tag of the target scene according to the new characteristics of the passive Internet of Things tag type and the characteristics of the target scene image.

[0011] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for selecting a passive IoT tag as described above is implemented.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for selecting a passive IoT tag as described in any one of the above is implemented.

[0013] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the method for selecting a passive IoT tag as described above.

[0014] The passive IoT tag selection method, device, equipment, medium and program product provided by the present invention, by taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map composed of the passive IoT tag type node, the passive IoT tag design scene node and the passive IoT tag packaging material node; inputting the node features in the passive IoT tag topology map into the passive IoT tag topology feature extractor composed of the graph attention neural network GAT, obtaining the passive IoT tag type new features output by the passive IoT tag topology feature extractor; inputting the target scene image of the passive IoT tag into the target scene image feature extractor composed of the large visual model LVM, obtaining the target scene image features output by the target scene image feature extractor; according to the passive IoT tag type new features and the target scene image features, determining the selection information of the passive IoT tag of the target scene. The present invention can accurately and efficiently realize the selection of passive IoT tags by combining the graph attention neural network and the large visual model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a flow chart of the method for selecting a passive IoT tag provided by the present invention.

[0017] Figure 2 It is a schematic diagram of the passive IoT tag topology diagram provided by the present invention.

[0018] Figure 3 It is a flow chart of the 5G-A passive IoT tag selection method based on a large-scale visual model provided by the present invention.

[0019] Figure 4 It is a structural schematic diagram of the selection device of the passive Internet of Things tag provided by the present invention.

[0020] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] In the related art, the selection of passive IoT tags is mainly based on the manual experience of experts, which is time-consuming, labor-intensive, inefficient, and inaccurate, thus affecting the promotion and application of 5G-A. Therefore, the present invention proposes to use the characteristics of the graph attention network that can assign different weights according to the differences in the influence of neighbor nodes in the graph network, and the ability to quickly extract image features of large-scale visual models, to build a multimodal graph attention network with a passive IoT tag type node as the center and a passive IoT target scene image as the common input.

[0023] Graph Attention Networks (GAT) introduces the attention mechanism into graph networks. It introduces the attention mechanism into the propagation layer, so that different weights are assigned to different adjacent nodes during the aggregation process of the central node features, and the attention to the neighboring nodes is different. The original graph convolutional network (GCN) and other graph neural networks use static and non-adaptive propagation rules, which cannot capture the neighboring nodes that contribute more to the classification of the central node. In real data, not all edges represent the same correlation strength. Unlike traditional neural networks, GAT will fully consider the relationship between data when processing input data, so that it can more accurately capture the correlation between data when processing graph structure data. The main advantage of GAT is its ability to automatically learn the relationship between nodes without manual preset. The core working principle of GAT is to calculate the relationship between nodes through the attention mechanism. In traditional neural networks, the state update of each node is performed independently, while in GAT, the state update of each node will take into account the state of its neighboring nodes. GAT will calculate the attention weight between a node and its neighboring nodes, and then update the state of the node according to this weight. The way of updating information by calculating weights enables GAT to better capture the structural information in the graph.

[0024] Large Vision Model (LVM) is a pure large vision model that does not require any natural language input or output and only uses image data for training and reasoning. Its goal is to learn general visual knowledge and adapt to different visual tasks and scenarios. The core idea of ​​LVM is to regard the image sequence as a high-dimensional time series signal, compress it into a low-dimensional vector sequence through an encoder, then use an autoregressive model to predict the vector of the next frame, and finally use a decoder to restore the predicted vector to the image. This method can utilize the redundant information in the image sequence, reduce the demand for training data, and improve the generalization and robustness of the model. LVM can achieve impressive performance on a variety of visual tasks, including image classification, object detection, image generation, image restoration, image style transfer, etc. It can even solve some graphic reasoning problems, such as judging the symmetry, rotation, quantity, position, etc. of the graphics, which are usually considered to require certain logical reasoning abilities rather than just visual perception abilities.

[0025] Combine the following Figure 1-Figure 5 The invention describes the selection method, device, equipment, medium and program product of the passive Internet of Things tag.

[0026] Figure 1FIG. 1 is a flow chart of the method for selecting a passive IoT tag provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101, taking a passive IoT tag type node as a central node, constructing a passive IoT tag topology map consisting of the passive IoT tag type node, the passive IoT tag design scenario node, and the passive IoT tag packaging material node.

[0027] It should be noted that the design scenario in the embodiment of the present invention can be understood as a design-oriented application scenario, and the design scenario node can be understood as a design-oriented application scenario node.

[0028] With the passive IoT tag type node as the central node, a passive IoT tag topology diagram is constructed, which consists of the passive IoT tag type node, the passive IoT tag design scenario node (i.e., the design oriented application scenario node), and the passive IoT tag packaging material node. Figure 2 The passive IoT tag topology graph can be expressed as G=(V, E), where V is the set of nodes and E is the set of edges. The nodes in the topology graph represent three types of heterogeneous data, i.e., three types of heterogeneous nodes. “Heterogeneous” refers to the heterogeneity of the types of nodes in the graph network of this scenario. The edges in the topology graph represent two relationships between the three types of heterogeneous data. The node feature set is represented by h, and the feature h of each vertex is i is a high-dimensional vector, and this scenario is regarded as a matching problem of the central nodes of a heterogeneous relationship graph.

[0029] The passive IoT tag topology diagram contains three types of heterogeneous nodes: (1) Passive IoT tag type: The characteristic of this type of node is the full name of the passive IoT tag type, such as license plate tags, asset tags, cable tags, etc., expressed as G={g 1 , g 2 , g 3 ,…,g n}; (2) Design for application scenarios: The characteristics of this type of node are the description of the application scenarios it is facing, such as asset management, personnel management, warehouse management, production line management, and environmental monitoring, etc., expressed as D = {d 1 ,d 2 ,d 3 ,…,d m}; (3) Packaging material: The characteristics of this type of node are the packaging material characteristics. The label has a variety of forms. The appropriate packaging material is selected according to the use environment (such as high temperature, humidity, chemical corrosion) to ensure the physical durability of the label. It is expressed as F = {f 1 、f 2 、f 3 ,…,f k}.

[0030] There are two types of relationships between nodes: 1) Between the passive IoT tag type and the application scenario it is designed for: the relationship that the passive IoT tag type is oriented towards a certain type of application scenario; 2) Between the passive IoT tag type and the packaging material: the relationship that the passive IoT tag type adopts a certain packaging material.

[0031] Step 102: input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT to obtain a new passive IoT tag type feature output by the passive IoT tag topology feature extractor.

[0032] refer to Figure 3 ,The passive IoT tag topology feature extractor includes a word embedding layer, a neighbor node level graph attention layer, and a center node level graph attention layer; the functions of each layer are as follows: The word embedding layer is used to convert the input node features into vector representations to obtain node feature vectors, and then input the node feature vectors into the neighbor node level graph attention layer. It can be understood that the main function of the word embedding layer is to map the original features of the nodes to a higher dimensional embedding space. It assigns a feature vector to each node and learns the representation of these node features through training. For example, the word embedding layer maps the original features of the nodes to a new feature space through a linear transformation. The specific operation is to multiply the input feature vector with a weight matrix to obtain a new feature vector for each node.

[0033] The neighbor node level graph attention layer is used to perform neighbor node level attention aggregation on the node feature vector, and input the aggregated feature vector obtained by aggregation into the central node level graph attention layer. It can be understood that the neighbor node level graph attention layer is responsible for calculating the attention coefficients between the node and its neighbor nodes, and aggregating the information of the neighbor nodes based on these coefficients. Specifically, this layer calculates the attention weight for the relationship between each node and its neighbor nodes, so that the contribution of the neighbor nodes is no longer equal, but adjusted according to the attention mechanism.

[0034] The central node-level graph attention layer is used to perform central node-level attention aggregation on the aggregated feature vector to obtain a new passive IoT tag type feature that simultaneously integrates multiple types of neighbor node features; neighbor nodes include design scenario nodes and packaging material nodes. It can be understood that the role of the central node-level graph attention layer is to learn the final representation of the central node by integrating the information of neighbor nodes. In graph neural networks, multiple layers of attention layers are usually stacked together. The central node-level graph attention layer is usually the last layer or near the last layer of the model. It further aggregates the neighbor information learned in the previous layers and finally outputs the representation of the node. In this layer, the model continues to apply the attention mechanism to weight the aggregated features, emphasizing the contribution of different neighbor nodes to the representation of the central node.

[0035] In general, the word embedding layer converts the original features of the node into a vector representation in the embedding space, the neighbor node-level graph attention layer calculates the attention coefficient between the node and its neighbors, and performs weighted aggregation on the features of the neighbor nodes. The center node-level graph attention layer further aggregates the information of the node in the multi-layer stacked GAT network to generate the final representation of the node.

[0036] Furthermore, the word embedding layer includes a first word embedding layer, a second word embedding layer and a third word embedding layer, and the neighbor node level graph attention layer includes a first neighbor node level graph attention layer and a second neighbor node level graph attention layer. Among them, the first word embedding layer is used to convert the input passive Internet of Things tag type feature into a vector representation to obtain a passive Internet of Things tag type feature vector, and input the passive Internet of Things tag type feature vector into the first neighbor node level graph attention layer and the second neighbor node level graph attention layer; the second word embedding layer is used to convert the input design scenario feature into a vector representation to obtain a design scenario feature vector, and input the design scenario feature vector into the first neighbor node level graph attention layer; the third word embedding layer is used to convert the input packaging material feature into a vector representation to obtain a packaging material feature vector, and input the packaging material feature vector into the second neighbor node level graph attention layer; the first neighbor node level graph attention layer is used to calculate the first attention weight of the design scenario node to the current central node according to the passive Internet of Things tag type feature vector and the design scenario feature vector, and perform weighted aggregation on the features of the design scenario node according to the first attention weight to obtain a first aggregated feature vector; the second neighbor node level graph attention layer is used to calculate the second attention weight of the packaging material node to the current central node according to the passive Internet of Things tag type feature vector and the packaging material feature vector, and perform weighted aggregation on the features of the neighbor nodes according to the attention weight to obtain a second aggregated feature vector.

[0037] The calculation of GAT is divided into two steps: calculating the attention coefficient and weighted summation. The GAT model is implemented by stacking graph attention layers, and the input of each graph attention layer is the text feature set generated by the node: ; in, is the text feature of the i-th node.

[0038] The output is a new text feature set: ; in, is the new text feature of the i-th node.

[0039] In order to calculate the weight of each neighbor node, a shared weight matrix W of F×F' is applied to each node, and then the attention coefficient can be calculated, which can represent the importance of node j relative to node i: ; Where W is the shared weight matrix, is the text feature of node i, is the text feature of node j, is the importance of node j relative to node i, is the attention coefficient.

[0040] In order to make the attention coefficient easier to calculate and compare, softmax is introduced to regularize the adjacent nodes j of all nodes i: ; in, is the importance of node j relative to node i, is the importance of node k relative to node i.

[0041] Therefore, the feature representation of node i after the graph attention layer is: that is, the new feature output by GAT for each vertex i (integrating neighborhood information).

[0042] ; in, is the activation function, N i is the set of neighbor nodes of node i, coefficient That is, the coefficient used for weighted summation during each convolution.

[0043] Step 103: input the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM to obtain the target scene image features output by the target scene image feature extractor.

[0044] Among them, the target scene image includes a target scene carrier morphology image and a target scene application environment image; the target scene image features include a target carrier morphology image feature vector and a target scene environment image feature vector.

[0045] LVM includes fully connected layers and pooling layers. The fully connected layer is used to map and convert the model's feature maps into specific categories or regression outputs, and the fully connected layer is used to further process the extracted high-level features so that the model can make final predictions. The pooling layer is used to reduce the spatial dimension of the features (for example, through maximum pooling or average pooling), thereby reducing the amount of computation, avoiding overfitting, and increasing the robustness of the model. The pooling operation helps to extract important features in the image without caring about the specific location.

[0046] It can be understood that the improved LVM adds fully connected layers and pooling layers on the basis of the existing large-scale visual model LVM weights, and only the added fully connected layers and pooling layers are trained to more flexibly adapt to the downstream specific visual task capabilities, and quickly train and identify the passive IoT target scene image features in this scene. Specifically, this improvement enables the original LVM to more flexibly adapt to downstream tasks through the newly added fully connected layers and pooling layers, such as image classification, object detection, image segmentation and other tasks, which may have different requirements in the feature space. By adding these layers, LVM can better transform and adapt to the needs of different tasks.

[0047] Step 104: Determine the selection information of the passive IoT tag of the target scene according to the new feature of the passive IoT tag type and the feature of the target scene image.

[0048] Specifically, the new features of the passive IoT tag type and the features of the target scene image are input into the passive IoT tag type matcher, and the selection information of the passive IoT tag of the target scene output by the passive IoT tag type matcher is obtained. Among them, the passive IoT tag type matcher includes a merging layer, a fully connected attention layer and an output layer; the merging layer is used to merge the feature vectors of the new features of the passive IoT tag type and the features of the target scene image, and input the merged feature vectors into the fully connected attention layer; the fully connected attention layer is used to determine the attention weights of each area in the target scene image according to the merged feature vector, and weight the image information of the target scene image based on the attention weights of each area to obtain the adjusted image information; the output layer is used to output the selection information of the passive IoT tag of the target scene based on the adjusted image information.

[0049] The selection method of the passive IoT tag provided by the embodiment of the present invention takes the passive IoT tag type node as the central node, constructs a passive IoT tag topology map composed of the passive IoT tag type node, the passive IoT tag design scene node and the passive IoT tag packaging material node; inputs the node features in the passive IoT tag topology map into the passive IoT tag topology feature extractor composed of the graph attention neural network GAT, and obtains the passive IoT tag type new features output by the passive IoT tag topology feature extractor; inputs the target scene image of the passive IoT tag into the target scene image feature extractor composed of the large visual model LVM, and obtains the target scene image features output by the target scene image feature extractor; and determines the selection information of the passive IoT tag of the target scene according to the passive IoT tag type new features and the target scene image features. The present invention can accurately and efficiently realize the selection of passive IoT tags by combining the graph attention neural network and the large visual model.

[0050] In order to further analyze and explain the selection method of the passive IoT tag proposed in the present invention, refer to the following examples.

[0051] At present, the selection of passive IoT tags is mainly based on the manual experience of experts, which is time-consuming, labor-intensive, inefficient, and inaccurate, thus affecting the promotion and application of 5G-A. Based on this, the embodiment of the present invention proposes a 5G-A passive IoT tag selection method based on a large-scale visual model. The method adds a fully connected layer and a pooling layer on the basis of the LVM weight of the large-scale visual model to more flexibly adapt to the downstream specific visual task capabilities, and quickly train and identify the passive IoT target scene image features of this scene; at the same time, the graph attention network is used to assign different weights according to the differences in the influence of neighbor nodes in the graph network, and build a passive IoT tag topology map centered on the passive IoT tag type node. Firstly, the passive IoT label type node performs neighbor node-level attention aggregation with its two adjacent neighbor nodes (design scenario nodes and packaging material nodes), and assigns different weights to the relationship between nodes according to the difference in importance. Then, the new features of the central node output by the central node-level attention layer are used together with the target scene image feature extractor composed of the fine-tuned pre-trained large-scale visual model LVM to extract the feature vectors of the target scene carrier morphological image and the target scene application environment image. Through the fully connected attention layer in the passive IoT label type matcher, it learns to assign larger attention weights to image areas that are more relevant to the passive IoT label. Finally, the matching relationship between the passive IoT label type central node and the target scene image features of the passive IoT label topology map is output, thereby realizing the precise adaptation and selection of 5G-A passive IoT labels and application scenarios.

[0052] refer to Figure 3, the 5G-A passive IoT tag selection process based on the large-scale visual model provided by the embodiment of the present invention is as follows: 1) From the passive IoT control platform, obtain in real time the passive IoT tag type, design application scenario, packaging material feature information and the relationship between the three with T as the time granularity, as well as the morphological image of the target scene carrier and the target scene application environment image.

[0053] 2) Taking the passive IoT tag type node as the central node, a passive IoT tag topology map is constructed, which consists of three heterogeneous nodes: passive IoT tag type, design-oriented application scenario, and packaging material.

[0054] 3.1) After encoding the characteristic information of passive IoT tag types, design-oriented application scenarios and packaging materials in the recent T time period into sequence representations, they are input into the passive IoT tag topology feature extractor composed of the graph attention neural network GAT; 3.2) After the feature contents of the three types of heterogeneous nodes are converted into fixed-length vectors through the word embedding layer, they are input into the neighbor node level graph attention layer respectively; 3.3) Taking the passive IoT tag type node as the central node, the attention aggregation is performed on the neighbor node level with the two adjacent neighbor nodes (design-oriented application scenario nodes and packaging material nodes), and different weights are assigned to the relationship between nodes according to the difference in importance; 3.4) Using their respective central node-level attention layers, output a new passive IoT tag type feature h that simultaneously integrates the features of two types of neighboring nodes gi '.

[0055] 4.1) After image preprocessing, the image of the target scene object and the target scene application environment image are cropped and the image pixel values ​​are normalized; 4.2) The preprocessed image sequence is input into the target scene image feature extractor composed of the fine-tuned pre-trained large-scale visual model LVM, and the target carrier morphological image feature vector and the target scene environment image feature vector are output. Among them, fine-tuning refers to adding fully connected layers and pooling layers based on the existing large-scale visual model LVM weights. Only the fully connected layers and pooling layers are trained to be more flexible to adapt to the downstream specific visual task capabilities, and quickly train and identify the passive IoT target scene image features in this scene.

[0056] 5) Add new feature h to passive IoT tag type gi', the target carrier morphological image feature vector and the target scene environment image feature vector are input to the passive IoT tag type matcher composed of a merging layer, a fully connected attention layer and an output layer. The fully connected attention layer in the passive IoT tag type matcher learns to assign a larger attention weight to the image area that is more relevant to the passive IoT tag, thereby learning the matching relationship between the passive IoT tag type center node of the passive IoT tag topology map and the target scene image features, and finally outputs the matching result between the target scene and the passive IoT tag type (1 for match, 0 for mismatch).

[0057] The model implementation of the embodiment of the present invention mainly includes the following two steps: Step 1: Topology map and visual data preprocessing: From the passive IoT management and control platform, the historical passive IoT tag type set, the design application scenario set and the packaging material feature set with T as the time granularity are obtained in real time, and a graph network composed of the three types of nodes is constructed according to the relationship between the three types of nodes. At the same time, the target scene data set is collected, where the target scene data set includes the target scene carrier morphology image set and the target scene application environment image set.

[0058] The passive IoT tag type set, design application scenario set, and packaging material feature set are encoded as sequence representations respectively. The encoding sequence length of each node feature is defined as F. The longest length F in the data set is taken as its encoding sequence length. The length of each data is filled with F, and the dictionary size is taken as z.

[0059] Target scene object morphology image set and target scene application environment image set: The resolution of the captured original image is 4096×2160, with 3 channels (RGB), so the size of a single input is 4096×2160×3. Since the input is too large and consumes a lot of computing power, and not all features are useful, the input after adjusting, cropping and processing the image using the OpenCV library is 800×800 pixels and is a single channel (gray scale). At the same time, in order to ensure that the data are in the same range, the data needs to be standardized. Each pixel value of each frame of the image is proportionally compressed to the range of 0 to 1, and then the pixel value of each frame of the image is subtracted from the pixel value of the global mean image to achieve normalization. The pixel value of the global mean image is obtained by calculating the mean of the pixel values ​​of each position of each frame of the training data. The pixel values ​​of the processed images will be normalized to a mean of 0 and a variance of 1.

[0060] The label matrix Y is whether each manually labeled passive IoT label type (M nodes) matches the target scene image (target scene carrier morphology image and target scene application environment image), and its shape is M. Finally, the total data set is divided into training data and test data. 80% of the entire data set is taken as training data, and the remaining 20% ​​is test data. The training set is used for training, and the test set is used to evaluate and verify the model.

[0061] Step 2: Build and train the model: This model builds a multimodal graph attention network with a passive IoT tag type node as the center and a target scene image as the common input. First, with the passive IoT tag type node as the center node, attention aggregation is performed with the two adjacent neighbor nodes (design-oriented application scenario nodes and packaging material nodes) at the neighbor node level. Then, the new passive IoT tag type features output by the respective center node level attention layers are used to learn the matching relationship between the passive IoT tag type center node of the passive IoT tag topology map and the target scene shape features through the passive IoT tag type matcher, and finally the matching results are output.

[0062] The model built is as follows Figure 3 As shown in the figure, the model mainly includes a passive IoT tag topology feature extractor, a target scene image feature extractor and a passive IoT tag type matcher.

[0063] (1) Passive IoT tag topology feature extractor: Input the characteristic attributes of three heterogeneous data: historical passive IoT tag type, design scenario (i.e. design oriented to application scenario), and packaging material characteristics with T as the time granularity.

[0064] Word embedding layer: Each word is converted into a vector using word embedding. The input data dimension is z, and the output is set to the 64-dimensional space vector that needs to be converted into the word. The output sequence length is F, so the shape of the output data of this layer is (None, F, 64). The function of this layer is to perform vector mapping on the input words and convert the index of each word into a 64-dimensional fixed-shape vector.

[0065] Branch 1: Learn the weight of the importance of the first type of neighbor node “design scenario” of passive IoT tag type i. Input the feature h of passive IoT tag type i gi , and its neighbor node design scene attribute characteristics h di , which are converted into vectors through the word embedding layer and then input into the neighbor node level graph attention layer, and the new feature h of the passive IoT label type after integrating the feature information of the neighbor node "design scenario" is output. gi 1 .

[0066] Branch 2: Learn the weight of the importance of the second type of neighbor node "packaging material" of passive IoT tag type i. Input the feature h of passive IoT tag type i gi , and the characteristics of the packaging materials of its neighboring nodes h fi , which are converted into vectors through the word embedding layer and then input into the neighbor node level graph attention layer, and the new passive IoT label type feature h is output after integrating the feature information of the neighbor node "packaging material". gi 2 .

[0067] Then, the new passive IoT tag type features after integrating the design scenario feature information and the new passive IoT tag type features after integrating the packaging material feature information output by the above two branches are input into the central node level graph attention layer for central node level attention aggregation, and the new passive IoT tag type features h that integrate the features of the two types of neighboring nodes are output. gi '.

[0068] Neighbor node-level graph attention layer: the number of convolution kernels is 256, and the activation function is set to "relu"; center node-level graph attention layer: the number of convolution kernels is 128, and the activation function is set to "relu".

[0069] (2) Target scene image feature extractor: The target scene object morphology image and the target scene application environment image are compressed to a size of 448×448 and then input into the large visual model LVM. The image is preprocessed using a pre-trained model to extract the feature representation of the image, and the image is encoded into a vector of fixed length. The last pooling layer of the model is used as the output, and the output shape is a 512×14×14 image feature vector matrix, where 14×14 is the number of image regions and 512 is the feature vector dimension of each image region.

[0070] (3) Passive IoT tag type matcher: Set the passive IoT tag type to a new feature h gi ', the target carrier morphological image feature vector and the target scene environment image feature vector are input into the passive IoT label type matcher composed of a merging layer, a fully connected attention layer and an output layer.

[0071] Merge layer: merge the new feature h of the passive IoT tag type gi ', the target carrier morphology image feature vector and the target scene environment image feature vector are merged.

[0072] Fully connected attention layer (Dense): By combining the information of the target scene image and the passive IoT label, an attention weight is generated for each area of ​​the image, and the image information is weighted, so as to focus on specific areas of the image and link the target scene image and the passive IoT label. The model learns to assign greater attention weights to image areas that are more relevant to the passive IoT label.

[0073] The target scene image feature matrix V and the passive IoT label text feature vector Q are input into the fully connected attention layer, where the addition of the matrix and the vector is to add each column of the matrix to the vector, and the final output is h a , and then h a Input to the softmax function and output the attention distribution of each area of ​​the image .

[0074] ; ; in, is the image feature vector weight, is the image feature vector bias, is the text feature vector weight, is the text feature vector bias, for Feature vector weights, for Eigenvector bias.

[0075] Calculate the attention weights based on their distribution With the image feature vector The sum of the products of : ; in, is the image feature vector, is the attention weight.

[0076] Output layer: It is composed of fully connected neurons, the number of neurons is set to 1, and the activation function is set to "sigmoid". Finally, the matching result between the target scene and the passive IoT tag type is output through the output layer (1 for match and 0 for mismatch).

[0077] Calculate the error between the predicted passive IoT tag selection result and the actual passive IoT tag selection matching result. The training goal is to minimize the error. The objective function selects the "binary_crossentropy" binary logarithmic loss function: ; in, This is the actual passive IoT tag selection and matching result. This is the predicted passive IoT tag selection and matching result.

[0078] Set the number of training rounds to 1000 (epochs=1000), and select the adam optimizer for the gradient descent optimization algorithm to improve the learning speed of traditional gradient descent (optimizer='adam'). The neural network can find the optimal weight value that minimizes the objective function through gradient descent, and the neural network will learn the weight value autonomously through training. Use the training set to train so that the objective function is as small as possible, and use the test set to evaluate and verify the model after each round of training. After the model converges, derive the weight of the model.

[0079] The 5G-A passive IoT tag selection method based on a large-scale visual model proposed in an embodiment of the present invention realizes the accurate adaptation and selection of 5G-A passive IoT tags and application scenarios.

[0080] The following is a description of a passive IoT tag selection device provided by the present invention. The passive IoT tag selection device described below and the passive IoT tag selection method described above can be referenced to each other.

[0081] refer to Figure 4 The passive IoT tag selection device provided by the present invention includes a construction module 401, a first feature extraction module 402, a second feature extraction module 403 and a selection module 404.

[0082] A construction module 401 is used to construct a passive IoT tag topology map consisting of a passive IoT tag type node, a design scenario node of a passive IoT tag, and a packaging material node of the passive IoT tag, with the passive IoT tag type node as a central node; The first feature extraction module 402 is used to input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT to obtain a new feature of the passive IoT tag type output by the passive IoT tag topology feature extractor; The second feature extraction module 403 is used to input the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM to obtain the target scene image features output by the target scene image feature extractor; The selection module 404 determines the selection information of the passive IoT tag of the target scene according to the new feature of the passive IoT tag type and the feature of the target scene image.

[0083] The passive IoT tag selection device provided by the embodiment of the present invention takes the passive IoT tag type node as the central node to construct a passive IoT tag topology map composed of the passive IoT tag type node, the passive IoT tag design scene node and the passive IoT tag packaging material node; the node features in the passive IoT tag topology map are input into the passive IoT tag topology feature extractor composed of the graph attention neural network GAT to obtain the passive IoT tag type new features output by the passive IoT tag topology feature extractor; the target scene image of the passive IoT tag is input into the target scene image feature extractor composed of the large visual model LVM to obtain the target scene image features output by the target scene image feature extractor; according to the passive IoT tag type new features and the target scene image features, the selection information of the passive IoT tag of the target scene is determined. The present invention can accurately and efficiently realize the selection of passive IoT tags by combining the graph attention neural network and the large visual model.

[0084] In one embodiment, the passive Internet of Things tag topology feature extractor includes a word embedding layer, a neighbor node level graph attention layer and a center node level graph attention layer; the word embedding layer is used to convert the input node features into vector representations to obtain node feature vectors, and input the node feature vectors into the neighbor node level graph attention layer; the neighbor node level graph attention layer is used to perform neighbor node level attention aggregation on the node feature vectors, and input the aggregated feature vectors obtained by aggregation into the center node level graph attention layer; the center node level graph attention layer is used to perform center node level attention aggregation on the aggregated feature vectors to obtain new passive Internet of Things tag type features that simultaneously integrate multiple types of neighbor node features; the neighbor nodes include the design scenario nodes and the packaging material nodes.

[0085] In one embodiment, the word embedding layer includes a first word embedding layer, a second word embedding layer and a third word embedding layer, and the neighbor node level graph attention layer includes a first neighbor node level graph attention layer and a second neighbor node level graph attention layer; the first word embedding layer is used to convert the input passive Internet of Things tag type feature into a vector representation to obtain a passive Internet of Things tag type feature vector, and input the passive Internet of Things tag type feature vector into the first neighbor node level graph attention layer and the second neighbor node level graph attention layer; the second word embedding layer is used to convert the input design scenario feature into a vector representation to obtain a design scenario feature vector, and input the design scenario feature vector into the first neighbor node level graph attention layer; the third word embedding layer is used to convert the input packaging material feature into a vector representation to obtain a design scenario feature vector, and input the design scenario feature vector into the first neighbor node level graph attention layer. is represented by a vector, a packaging material feature vector is obtained, and the packaging material feature vector is input into the second neighbor node level graph attention layer; the first neighbor node level graph attention layer is used to calculate the first attention weight of the design scenario node to the current central node according to the passive Internet of Things tag type feature vector and the design scenario feature vector, and weightedly aggregate the features of the design scenario node according to the first attention weight to obtain a first aggregated feature vector; the second neighbor node level graph attention layer is used to calculate the second attention weight of the packaging material node to the current central node according to the passive Internet of Things tag type feature vector and the packaging material feature vector, and weightedly aggregate the features of the neighbor nodes according to the attention weight to obtain a second aggregated feature vector.

[0086] In one embodiment, the target scene image includes a target scene carrier morphology image and a target scene application environment image; the target scene image features include a target carrier morphology image feature vector and a target scene environment image feature vector; the LVM includes a fully connected layer and a pooling layer.

[0087] In one embodiment, the selection module 404 is specifically used to: The new passive IoT tag type feature and the target scene image feature are input into a passive IoT tag type matcher to obtain the selection information of the passive IoT tag of the target scene output by the passive IoT tag type matcher.

[0088] In one embodiment, the passive IoT tag type matcher includes a merging layer, a fully connected attention layer and an output layer; the merging layer is used to merge the new features of the passive IoT tag type and the target scene image features into feature vectors, and input the merged feature vectors into the fully connected attention layer; the fully connected attention layer is used to determine the attention weights of each area in the target scene image based on the merged feature vectors, and weight the image information of the target scene image based on the attention weights of each area to obtain adjusted image information; the output layer is used to output the selection information of the passive IoT tag of the target scene based on the adjusted image information.

[0089] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the selection method of the passive IoT tag, which includes: taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map composed of the passive IoT tag type node, the design scene node of the passive IoT tag, and the packaging material node of the passive IoT tag; inputting the node features in the passive IoT tag topology map into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT, and obtaining the new passive IoT tag type features output by the passive IoT tag topology feature extractor; inputting the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM, and obtaining the target scene image features output by the target scene image feature extractor; and determining the selection information of the passive IoT tag of the target scene according to the new passive IoT tag type features and the target scene image features.

[0090] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0091] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the selection method of the passive IoT tag provided by the above methods, the method including: taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map composed of the passive IoT tag type node, the design scene node of the passive IoT tag, and the packaging material node of the passive IoT tag; inputting the node features in the passive IoT tag topology map into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT, and obtaining the new passive IoT tag type features output by the passive IoT tag topology feature extractor; inputting the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM, and obtaining the target scene image features output by the target scene image feature extractor; and determining the selection information of the passive IoT tag of the target scene according to the new passive IoT tag type features and the target scene image features.

[0092] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the selection method for the passive IoT tag provided by the above-mentioned methods, the method comprising: taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map consisting of the passive IoT tag type node, the design scene node of the passive IoT tag, and the packaging material node of the passive IoT tag; inputting the node features in the passive IoT tag topology map into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT, and obtaining new features of the passive IoT tag type output by the passive IoT tag topology feature extractor; inputting the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM, and obtaining the target scene image features output by the target scene image feature extractor; and determining the selection information of the passive IoT tag of the target scene according to the new features of the passive IoT tag type and the target scene image features.

[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for selecting a passive IoT tag, characterized in that: include: Taking the passive IoT tag type node as the central node, constructing a passive IoT tag topology map consisting of the passive IoT tag type node, the design scenario node of the passive IoT tag, and the packaging material node of the passive IoT tag; Input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT, and obtain a new passive IoT tag type feature output by the passive IoT tag topology feature extractor; Inputting the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM, and obtaining the target scene image features output by the target scene image feature extractor; According to the new features of the passive Internet of Things tag type and the target scene image features, selection information of the passive Internet of Things tag of the target scene is determined.

2. The method for selecting a passive IoT tag according to claim 1, characterized in that: The passive IoT tag topology feature extractor includes a word embedding layer, a neighbor node level graph attention layer, and a center node level graph attention layer; The word embedding layer is used to convert the input node features into vector representations to obtain node feature vectors, and input the node feature vectors into the neighbor node level graph attention layer; The neighbor node level graph attention layer is used to perform neighbor node level attention aggregation on the node feature vector, and input the aggregated feature vector obtained by aggregation into the central node level graph attention layer; The central node-level graph attention layer is used to perform central node-level attention aggregation on the aggregated feature vector to obtain a new passive IoT tag type feature that simultaneously integrates features of multiple types of neighboring nodes; the neighboring nodes include the design scenario node and the packaging material node.

3. The method for selecting a passive IoT tag according to claim 2, characterized in that: The word embedding layer includes a first word embedding layer, a second word embedding layer and a third word embedding layer, and the neighbor node level graph attention layer includes a first neighbor node level graph attention layer and a second neighbor node level graph attention layer; The first word embedding layer is used to convert the input passive Internet of Things tag type feature into a vector representation to obtain a passive Internet of Things tag type feature vector, and input the passive Internet of Things tag type feature vector into the first neighbor node level graph attention layer and the second neighbor node level graph attention layer; The second word embedding layer is used to convert the input design scene feature into a vector representation to obtain a design scene feature vector, and input the design scene feature vector into the first neighbor node level graph attention layer; The third word embedding layer is used to convert the input packaging material feature into a vector representation to obtain a packaging material feature vector, and input the packaging material feature vector into the second neighbor node level graph attention layer; The first neighbor node level graph attention layer is used to calculate the first attention weight of the design scenario node to the current central node according to the passive IoT tag type feature vector and the design scenario feature vector, and perform weighted aggregation on the features of the design scenario node according to the first attention weight to obtain a first aggregated feature vector; The second neighbor node level graph attention layer is used to calculate the second attention weight of the packaging material node to the current central node based on the passive IoT tag type feature vector and the packaging material feature vector, and perform weighted aggregation on the features of the neighbor nodes according to the attention weight to obtain a second aggregated feature vector.

4. The method for selecting a passive IoT tag according to claim 1, characterized in that: The target scene image includes a target scene carrier morphology image and a target scene application environment image; the target scene image features include a target carrier morphology image feature vector and a target scene environment image feature vector; the LVM includes a fully connected layer and a pooling layer.

5. The method for selecting a passive IoT tag according to claim 1, characterized in that: The step of determining the selection information of the passive IoT tag of the target scene according to the new feature of the passive IoT tag type and the feature of the target scene image includes: The new passive IoT tag type feature and the target scene image feature are input into a passive IoT tag type matcher to obtain the selection information of the passive IoT tag of the target scene output by the passive IoT tag type matcher.

6. The method for selecting a passive IoT tag according to claim 5, characterized in that: The passive IoT tag type matcher includes a merging layer, a fully connected attention layer, and an output layer; The merging layer is used to merge the new feature of the passive IoT tag type and the target scene image feature into feature vectors, and input the merged feature vectors into the fully connected attention layer; The fully connected attention layer is used to determine the attention weight of each region in the target scene image according to the merged feature vector, and weight the image information of the target scene image based on the attention weight of each region to obtain adjusted image information; The output layer is used to output the selection information of the passive IoT tag of the target scene based on the adjusted image information.

7. A device for selecting a passive IoT tag, characterized in that: include: A construction module is used to construct a passive IoT tag topology map consisting of a passive IoT tag type node, a design scenario node of a passive IoT tag, and a packaging material node of the passive IoT tag, with the passive IoT tag type node as a central node; A first feature extraction module is used to input the node features in the passive IoT tag topology graph into a passive IoT tag topology feature extractor composed of a graph attention neural network GAT to obtain a new feature of the passive IoT tag type output by the passive IoT tag topology feature extractor; A second feature extraction module is used to input the target scene image of the passive IoT tag into a target scene image feature extractor composed of a large visual model LVM to obtain the target scene image features output by the target scene image feature extractor; A selection module is used to determine the selection information of the passive Internet of Things tag of the target scene according to the new characteristics of the passive Internet of Things tag type and the characteristics of the target scene image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method for selecting a passive IoT tag according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for selecting a passive IoT tag according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for selecting a passive IoT tag according to any one of claims 1 to 6 is implemented.

Images