A part category identification method and system based on deep learning and a knowledge graph

Abstract

This invention discloses a method and system for identifying parts categories based on deep learning and knowledge graphs. The method involves acquiring a dataset of parts images, preprocessing and augmenting the dataset, constructing and training a recognition model, extracting and saving features from the parts images using the trained model, and deriving feature vectors for different parts images. For the parts images to be identified, the model extracts feature information and compares it with the feature vectors of existing images to obtain the most similar result. Based on the comparison result, a query is automatically used to search for corresponding information in the knowledge graph and the result is returned. By identifying parts images and obtaining detailed information from the knowledge graph, this method can be useful in equipment maintenance and other fields. This invention rapidly returns a large amount of information from a single image, effectively accelerating information retrieval in scenarios such as equipment maintenance, parts replacement, and automated repair, and provides a valuable reference for improving the efficiency of parts information retrieval.

Description

Technical Field

[0001] This invention belongs to the fields of equipment operation and maintenance, knowledge graphs and deep learning, and relates to a method and system for identifying part categories based on deep learning and knowledge graphs. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, higher demands are being placed on the identification and management of parts. Traditional parts identification methods often rely on manual inspection or simple image processing techniques, which have limitations in efficiency and accuracy. To improve the efficiency and accuracy of parts identification, deep learning technology has been introduced into the field, achieving fast and accurate parts identification by automatically extracting image features and performing intelligent matching.

[0003] In the feature extraction process of part images, neural networks can identify information such as edges, textures, and shapes in the image and transform them into high-dimensional feature vectors. These feature vectors can characterize the key attributes of the part, providing a basis for subsequent similarity comparison. Image feature matching is the process of comparing the feature vectors of the input image with feature vectors stored in a database. By calculating the distance or similarity between two feature vectors, the part image most similar to the input image can be found.

[0004] A knowledge graph is a structured semantic knowledge base that stores relationships between entities in the form of a graph. In a parts identification system, a knowledge graph can be used to store relevant information about parts, such as their name, specifications, and uses. By querying the knowledge graph, users can quickly retrieve relevant information based on the characteristics or attributes of the parts, including part descriptions, potential problems, and solutions, providing feedback to the user.

[0005] Existing part classification and recognition technologies typically use a Softmax layer as the final layer of a neural network to obtain classification and recognition results. This method works well when there are few types of parts, but its accuracy is difficult to guarantee when there are many types of parts. Furthermore, this method requires specialized model training based on the number of part types and the part image data in the dataset. When the dataset changes or the types of parts to be recognized change, the model needs to be retrained, resulting in poor scalability. Additionally, the data returned by this method only contains the recognition results and lacks more detailed and structured information feedback. Summary of the Invention

[0006] The purpose of this invention is to solve the problems in the prior art and provide a method and system for identifying part categories based on deep learning and knowledge graphs.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] This invention proposes a part category identification method based on deep learning and knowledge graphs, comprising:

[0009] Obtain the image of the part to be queried and the preprocessed image of the part;

[0010] Based on the VGG11 model, a part image recognition model is established. The preprocessed part image is input into the part image recognition model to obtain the trained image recognition model.

[0011] The trained image recognition model is used to extract features from the preprocessed part images to obtain the feature vectors contained in different part images;

[0012] Input the image of the part to be queried into the trained image recognition model to obtain the feature vector;

[0013] By processing the feature vectors and the feature vectors contained in different part images, detailed information about the parts can be obtained, thus enabling part category recognition.

[0014] Preferably, the preprocessed part image is obtained, specifically as follows:

[0015] The part images are sequentially filtered, cropped, scaled, and expanded to obtain preprocessed part images.

[0016] Preferably, the step of establishing a part image recognition model based on the VGG11 model specifically involves:

[0017] The part image recognition model adds a BN layer after each convolutional layer of the VGG11 model, and concatenates the outputs of the first two of the three fully connected layers in the VGG11 model into a 2048-dimensional array, which is then used as the input to the third fully connected layer.

[0018] Preferably, the part image recognition model has 5 convolutional blocks, a total of 8 convolutional layers, and 3 fully connected layers. The outputs of the first two fully connected layers are concatenated as the input of the third fully connected layer, and the output of the last fully connected layer is the classification result during model training. Each convolutional block is followed by a max pooling layer, each convolutional layer is followed by a BN layer and a ReLU activation function, and each fully connected layer is followed by a ReLU activation function and a Dropout layer. The feature vector concatenated from the first two fully connected layers is taken as the extracted image features, and the recognition result is obtained by comparing the image features during use.

[0019] Preferably, the step of processing the feature vectors and the feature vectors contained in different part images to obtain detailed part information and realize part category recognition specifically involves:

[0020] The cosine similarity between the feature vector and the feature vector contained in different part images is obtained sequentially. The image corresponding to the result with the highest similarity is taken as the search result. Based on the search result, the detailed information of the part is obtained from the knowledge graph.

[0021] This invention proposes a parts category recognition system based on deep learning and knowledge graphs, comprising:

[0022] An image acquisition module is used to acquire the image of the part to be queried and the preprocessed image of the part.

[0023] The model building module is used to establish a part image recognition model based on the VGG11 model, input the preprocessed part image into the part image recognition model, and obtain the trained image recognition model.

[0024] The feature extraction module is used to extract features from the preprocessed part images using a trained image recognition model, and to obtain feature vectors contained in different part images;

[0025] The image processing module is used to input the image of the part to be queried into the trained image recognition model to obtain feature vectors;

[0026] The data processing module is used to process feature vectors and feature vectors contained in different part images to obtain detailed part information and realize part category recognition.

[0027] Preferably, the step of establishing a part image recognition model based on the VGG11 model specifically involves:

[0028] The part image recognition model adds a BN layer after each convolutional layer of the VGG11 model, and concatenates the outputs of the first two of the three fully connected layers in the VGG11 model into a 2048-dimensional array, which is then used as the input to the third fully connected layer.

[0029] Preferably, the step of processing the feature vectors and the feature vectors contained in different part images to obtain detailed part information and realize part category recognition specifically involves:

[0030] The cosine similarity between the feature vector and the feature vector contained in different part images is obtained sequentially. The image corresponding to the result with the highest similarity is taken as the search result. Based on the search result, the detailed information of the part is obtained from the knowledge graph.

[0031] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a part category identification method based on deep learning and knowledge graph.

[0032] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a part category identification method based on deep learning and knowledge graphs.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] This invention proposes a part category recognition method based on deep learning and knowledge graphs. Compared to using image classification models for part recognition, this invention uses neural networks for image feature extraction and feature comparison, significantly increasing the number of part categories that can be recognized compared to general image classification models. Furthermore, it eliminates the need to retrain the model when adding new part categories, demonstrating good scalability. The addition of feedback based on the most similar image effectively provides feedback to the user, facilitating manual confirmation of the recognition results. Using a knowledge graph as a source of detailed information increases the amount of information in the feedback results and improves the structure of the feedback information. By quickly providing the large amount of information required from a single image, it effectively accelerates information retrieval speed in scenarios such as equipment maintenance, parts replacement, and automated repair, offering valuable reference for enterprises to improve the efficiency of parts information retrieval in various scenarios. It does not rely on model classification results, but uses feature comparison to find the most similar result; new part categories can be added directly without retraining; it provides more detailed knowledge feedback; and it automatically obtains knowledge feedback based on comparison results, without requiring accurate prediction of the part name.

[0035] This invention proposes a parts category recognition system based on deep learning and knowledge graphs. By dividing the system into an image acquisition module, a model building module, a feature extraction module, an image processing module, and a data processing module, it acquires parts category information and achieves parts category recognition. The modular approach ensures that each module is independent, facilitating unified management of all modules. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the part category identification method based on deep learning and knowledge graph of the present invention.

[0038] Figure 2 This is a detailed flowchart of the identification method of the present invention.

[0039] Figure 3 This is a structural diagram of the part image recognition model based on the VGG11 model of the present invention, where the maxpool layer is the end of each convolutional block.

[0040] Figure 4 This is an example of the knowledge feedback obtained from image recognition results according to the present invention.

[0041] Figure 5 This is a data format diagram used for the knowledge graph of this invention.

[0042] Figure 6 This is a diagram of the part category recognition system based on deep learning and knowledge graphs of the present invention.

[0043] Figure 7 This is a schematic diagram of the structure of an electronic device according to the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0047] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0048] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0049] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0050] The present invention will now be described in further detail with reference to the accompanying drawings:

[0051] This invention proposes a part category identification method based on deep learning and knowledge graphs, such as... Figure 1 As shown, it includes:

[0052] S1. Obtain the image of the part to be queried and the preprocessed image of the part;

[0053] Obtain the preprocessed part image, specifically:

[0054] The part images are sequentially filtered, cropped, scaled, and expanded to obtain preprocessed part images.

[0055] S2. Based on the VGG11 model, establish a part image recognition model. Input the preprocessed part image into the part image recognition model to obtain the trained image recognition model.

[0056] The establishment of a part image recognition model based on the VGG11 model is specifically as follows:

[0057] The part image recognition model adds a BN layer after each convolutional layer of the VGG11 model, and concatenates the outputs of the first two of the three fully connected layers in the VGG11 model into a 2048-dimensional array, which is then used as the input to the third fully connected layer.

[0058] The part image recognition model has 5 convolutional blocks, a total of 8 convolutional layers, and 3 fully connected layers. The outputs of the first two fully connected layers are concatenated as the input of the third fully connected layer, and the output of the last fully connected layer is the classification result during model training. Each convolutional block is followed by a max pooling layer, each convolutional layer is followed by a batch normalization (BN) layer and a ReLU activation function, and each fully connected layer is followed by a ReLU activation function and a Dropout layer. The feature vector concatenated from the first two fully connected layers is taken as the extracted image features, and the recognition result is obtained by comparing the image features.

[0059] S3. Use the trained image recognition model to extract features from the preprocessed part images and obtain the feature vectors contained in different part images;

[0060] S4. Input the image of the part to be queried into the trained image recognition model to obtain the feature vector;

[0061] S5. Process the feature vectors and the feature vectors contained in different part images to obtain detailed part information and realize part category recognition.

[0062] The process of processing feature vectors and feature vectors contained in different part images to obtain detailed part information and achieve part category recognition specifically involves:

[0063] The cosine similarity between the feature vector and the feature vector contained in different part images is obtained sequentially. The image corresponding to the result with the highest similarity is taken as the search result. Based on the search result, the detailed information of the part is obtained from the knowledge graph.

[0064] The following is combined with Figure 2 The method is described in detail below:

[0065] Step 1: Data Construction: Collect images of various parts from multiple angles and models to form a dataset;

[0066] The dataset can be a publicly available dataset or a dataset constructed by collecting images from the internet. In practical application scenarios, it is advisable to use real-world photos. When constructing a dataset by collecting images from the internet or by taking photos yourself, you need to name each image file according to the format of "part name + number" based on the image content, or use a table to save the images and "part name + number" information one by one.

[0067] Knowledge graph construction mainly includes the creation of component entities and the creation of relationships between entities. Based on recognition requirements and information feedback needs, the created entity information mainly includes numbers, names, and various attributes that need to be fed back; while the creation of relationships is determined based on the specific connections between components.

[0068] Step 2, Data Preprocessing: Filter the data obtained in Step 1, remove images that obviously do not meet the requirements, crop all images according to a fixed aspect ratio, scale them to a uniform resolution that meets the requirements of deep learning models, and divide the dataset into training and test sets according to a certain ratio.

[0069] This includes filtering, cropping, scaling, and augmenting the images in the dataset. The specific process is as follows:

[0070] Step 201: Filtering. Based on the model input requirements, delete images that are clearly not clear enough, have distorted brightness, or are not fully displayed by the shooting device. Especially when choosing to train a self-made model, such data is considered noisy and will affect the training of the deep learning model.

[0071] Step 202: Cropping. Based on the model input requirements, select an appropriate aspect ratio and crop all images that do not conform to this uniform aspect ratio, extracting their key parts for subsequent processing.

[0072] Step 203: Scaling. According to the input requirements of the deep learning model, all images in the dataset are scaled to a uniform resolution. This invention uses a 224*224 pixel parts image dataset.

[0073] Step 204: Expansion. The dataset is expanded by flipping, rotating, and adjusting the brightness of the images. The new data obtained through these methods is often of lower quality than the original images. This step can be omitted if the prepared dataset has sufficient data.

[0074] Step 3: Deep learning model setup: Build an image recognition model based on the VGG11 model and train the image recognition model.

[0075] Step 301: This invention selects to construct an image recognition model based on an improved VGG11. The reason for choosing VGG11 as the basis for optimization compared to the VGG16 model is that the VGG11 model has a much smaller number of parameters, which is beneficial for project miniaturization; and the overfitting phenomenon of VGG11 is significantly reduced compared to the former. The improvements are: adding a BN (Batch Normalization) layer after each convolutional layer of the VGG11 model to alleviate overfitting during model training; and concatenating the outputs of the first two (both 1024-dimensional) of the three fully connected layers in the model into a 2048-dimensional array and using it as the input to the third fully connected layer, thereby obtaining more feature information, which is more conducive to subsequent feature extraction and image recognition. For the detailed structure of the model, please refer to the appendix. Figure 3 .

[0076] Specifically, the improved model based on VGG11 has 5 convolutional blocks, a total of 8 convolutional layers, and 3 fully connected layers. The outputs of the first two fully connected layers are concatenated as the input to the third fully connected layer, and the output of the last fully connected layer is the classification result during model training. Each convolutional block is followed by a max pooling layer, each convolutional layer is followed by a batch normalization (BN) layer and a ReLU activation function, and each fully connected layer is followed by a ReLU activation function and a Dropout layer.

[0077] In this system, all convolutional layers have a kernel size of 3*3 and a padding parameter of 1 (i.e., the number of zeros padded around the data to ensure sufficient space between the element content and the border, guaranteeing that the data size of each channel remains unchanged after convolution); all max pooling layers have a window size of 2*2 and a stride of 2 (i.e., the size of each side of each channel is halved); Dropout is a regularization technique to prevent overfitting in deep neural networks, which increases the model's generalization ability by randomly shutting down some neurons during training. The ReLU activation function works as follows:

[0078]

[0079] When image data of size 224*224*3 (224*224 pixels, 3 channels, and so on) passes through the first convolutional block, the feature map size becomes 112*112*64; after the second convolutional block, the size becomes 56*56*128; after the third convolutional block, the size becomes 28*28*256; after the fourth convolutional block, the size becomes 14*14*512; after the fifth convolutional block, the size becomes 7*7*512; after the first fully connected layer, the size becomes 1*1*1024; after the second fully connected layer, the size becomes 1*1*1024; after the concatenation operation, the size becomes 1*1*2048; and after the last fully connected layer, the size becomes 1*1*50. The number of channels corresponds to the number of categories in the classification task.

[0080] Step 302: Obtain model parameters. Train and compare the VGG11, the VGG11 with added BN layers, and the model built in S301 on the same device with the same hyperparameters. Then save the model parameters for feature extraction in steps 4 and 5.

[0081] (1) Dataset partitioning. Using the device image dataset prepared in S1, perform the data preprocessing in S2, and then partition the dataset into training and test sets according to a certain ratio. The model is first trained on the training set, and then debugged and improved on the test set.

[0082] (2) Model hyperparameter settings. When starting to train the model, the main training hyperparameters to set include: number of training epochs, batch size, image size (Img-size), learning rate, and dropout value. At the same time, the training optimizer should be selected as Adam, and the loss function should be the cross-entropy loss function.

[0083] Table 1. Hyperparameter Configuration

[0084] Epochs 150 Batch-size 32 Img-size 224*224*3 Learning_rate 1e-4 (reduced to 0.92x every four rounds) Dropout 0.5 Optimizer Adam loss function Cross-entropy loss function

[0085] The cross-entropy loss function is defined as follows:

[0086]

[0087] Where n is the total number of tags, P(X) i Let Q(X) be the probability of class i in the true distribution. i ) represents the probability of the i-th class in the prediction results.

[0088] The cross-entropy loss function in the PyTorch library includes a Softmax layer to convert the result into a probabilistic form, defined as follows:

[0089]

[0090] Where y i Let be the output value of the i-th node, and m be the number of output nodes, i.e., the number of categories. The Softmax function can be used to convert the output values ​​of multi-class classification into probability values ​​for each possible category, representing the likelihood of belonging to each category.

[0091] The Adam optimizer adjusts the learning rate of each parameter by calculating the first and second moment estimates of the gradient, thereby achieving more efficient network training.

[0092] (3) Model training. The model was trained using the Python programming language and underwent epochs of training. The model converged, and its performance improved significantly. The accuracy on the test set reached the expected level.

[0093] (4) Model Comparison. After multiple rounds of thorough training, the best performance of each model was compared. The VGG11 model achieved a maximum accuracy of 71.2% on the test set, while the training set accuracy was 99.9%, indicating significant overfitting. The VGG11 model with BN layer achieved a maximum accuracy of 84.0% on the test set, while the training set accuracy was 89.9%, showing better performance compared to the VGG11 model without BN layer. The optimized model achieved a maximum accuracy of 89.2% on the test set, outperforming the ordinary VGG11 model.

[0094] Finally, save the model parameters.

[0095] Step 4: Feature Extraction from Dataset Images: Use the trained model and model parameters to extract features from all images in the dataset. The specific method is as follows:

[0096] Load the trained model and parameters, remove the last fully connected layer, input the images from the dataset into the model, retain the outputs of the first two fully connected layers, and concatenate them into a 2048-dimensional feature vector. The resulting feature array containing the feature vectors is saved as key-value pairs in an npz file, where the key is the filename of each image (a combination of part name and number). This completes the preparation for image recognition.

[0097] Step 5, Feature Comparison: For the part images to be queried, feature vectors are extracted using the model selected in S3 and the trained parameters. These feature vectors are then compared with the data saved in Step 4 to select the most similar images.

[0098] The saved model and parameters are used to extract feature vectors from the parts images to be queried. These feature vectors are then compared sequentially with all data in the npz file to obtain cosine similarity. The image with the highest similarity is selected as the search result, and this result is output for user verification, reducing the risk of errors. The cosine similarity calculation formula is shown below:

[0099]

[0100] Step 6: Query the knowledge graph: Based on the comparison results obtained in Step 5, retrieve information about the part from the knowledge graph to support part information retrieval needs in different scenarios, such as... Figure 4 As shown.

[0101] Based on the image names corresponding to the search results, the part names and other information are obtained. The Cypher query language is then used to retrieve detailed information about the part from the knowledge graph of the neo4j graph database and provide the results to the user. Specifically: the part name can be obtained from the key names in the search results; the knowledge graph in neo4j needs to be pre-built, storing the required part information and relationship information. The data format used to build the knowledge graph is as follows: Figure 5 As shown.

[0102] This invention proposes a part category recognition method based on deep learning and knowledge graphs, addressing the problems of long recognition time and low accuracy in related technologies for part recognition and retrieval. It also incorporates knowledge graphs to provide feedback on relevant detailed information. Compared to using image classification models for part recognition, this invention uses neural networks for image feature extraction and comparison, significantly increasing the number of part categories that can be recognized. Furthermore, it eliminates the need for model retraining when the dataset changes (e.g., adding new part categories), demonstrating good scalability. The addition of feedback based on the most similar image effectively provides feedback to the user, facilitating manual confirmation of the recognition results. Using a knowledge graph as the source of detailed information increases the information content and improves the structure of the feedback. Compared to directly querying the knowledge graph, this invention adds an image detection step, allowing for the retrieval of information on parts not previously known, thus expanding its application scenarios. It can utilize both public and self-built datasets, facilitating expansion and portability. By filtering, cropping, scaling, and expanding images, it achieves standardized input images for the model, increasing the data volume of the model training dataset. The combination with knowledge graphs effectively improves the density and reliability of the feedback information.

[0103] Example 2

[0104] This invention proposes a parts category recognition system based on deep learning and knowledge graphs, such as... Figure 6 As shown, it includes:

[0105] An image acquisition module is used to acquire the image of the part to be queried and the preprocessed image of the part.

[0106] Obtain the preprocessed part image, specifically:

[0107] The part images are sequentially filtered, cropped, scaled, and expanded to obtain preprocessed part images.

[0108] The model building module is used to establish a part image recognition model based on the VGG11 model, input the preprocessed part image into the part image recognition model, and obtain the trained image recognition model.

[0109] The establishment of a part image recognition model based on the VGG11 model is specifically as follows:

[0110] The part image recognition model adds a BN layer after each convolutional layer of the VGG11 model, and concatenates the outputs of the first two of the three fully connected layers in the VGG11 model into a 2048-dimensional array, which is then used as the input to the third fully connected layer.

[0111] The part image recognition model has 5 convolutional blocks, a total of 8 convolutional layers, and 3 fully connected layers. The outputs of the first two fully connected layers are concatenated as the input of the third fully connected layer, and the output of the last fully connected layer is the classification result during model training. Each convolutional block is followed by a max pooling layer, each convolutional layer is followed by a batch normalization (BN) layer and a ReLU activation function, and each fully connected layer is followed by a ReLU activation function and a Dropout layer. The feature vector concatenated from the first two fully connected layers is taken as the extracted image features, and the recognition result is obtained by comparing the image features.

[0112] The feature extraction module is used to extract features from the preprocessed part images using a trained image recognition model, and to obtain feature vectors contained in different part images;

[0113] The image processing module is used to input the image of the part to be queried into the trained image recognition model to obtain feature vectors;

[0114] The data processing module is used to process feature vectors and feature vectors contained in different part images to obtain detailed part information and realize part category recognition.

[0115] The process of processing feature vectors and feature vectors contained in different part images to obtain detailed part information and achieve part category recognition specifically involves:

[0116] The cosine similarity between the feature vector and the feature vector contained in different part images is obtained sequentially. The image corresponding to the result with the highest similarity is taken as the search result. Based on the search result, the detailed information of the part is obtained from the knowledge graph.

[0117] Example 3

[0118] Please see Figure 7 As shown, the present invention also provides an electronic device 100 based on a part category identification method using deep learning and knowledge graphs; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0119] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the deep learning and knowledge graph-based part category identification method described in Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0120] The at least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor. The processor 102 is the control center of the electronic device 100, connecting various parts of the electronic device 100 via various interfaces and lines.

[0121] The memory 101 in the electronic device 100 stores multiple instructions to implement a part category recognition method based on deep learning and knowledge graphs, and the processor 102 can execute the multiple instructions to achieve the following:

[0122] Obtain the image of the part to be queried and the preprocessed image of the part;

[0123] Based on the VGG11 model, a part image recognition model is established. The preprocessed part image is input into the part image recognition model to obtain the trained image recognition model.

[0124] The trained image recognition model is used to extract features from the preprocessed part images to obtain the feature vectors contained in different part images;

[0125] Input the image of the part to be queried into the trained image recognition model to obtain the feature vector;

[0126] By processing the feature vectors and the feature vectors contained in different part images, detailed information about the parts can be obtained, thus enabling part category recognition.

[0127] Example 4

[0128] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0129] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for identifying part categories based on deep learning and knowledge graphs, characterized in that, include: Obtain the image of the part to be queried and the preprocessed image of the part; Based on the VGG11 model, a part image recognition model is established. The preprocessed part image is input into the part image recognition model to obtain the trained image recognition model. The trained image recognition model is used to extract features from the preprocessed part images to obtain the feature vectors contained in different part images; Input the image of the part to be queried into the trained image recognition model to obtain the feature vector; By processing the feature vectors and the feature vectors contained in different part images, detailed information about the parts can be obtained, thereby achieving part category recognition; The establishment of a part image recognition model based on the VGG11 model is specifically as follows: The part image recognition model adds a BN layer after each convolutional layer of the VGG11 model. The outputs of the first two fully connected layers in the VGG11 model are concatenated into a 2048-dimensional array and used as the input to the third fully connected layer. The model has five convolutional blocks (a total of eight convolutional layers) and three fully connected layers. The outputs of the first two fully connected layers are concatenated as the input to the third fully connected layer, and the output of the last fully connected layer is the classification result during model training. Each convolutional block is followed by a max pooling layer, each convolutional layer by a BN layer and a ReLU activation function, and each fully connected layer by a ReLU activation function and a Dropout layer. The feature vector concatenated from the first two fully connected layers is taken as the extracted image features, and the recognition result is obtained by comparing these features. The process of processing feature vectors and feature vectors contained in different part images to obtain detailed part information and achieve part category recognition specifically involves: The cosine similarity between the feature vector and the feature vector contained in different part images is obtained sequentially. The image corresponding to the result with the highest similarity is taken as the search result. Based on the search result, the detailed information of the part is obtained from the knowledge graph.

2. The part category identification method based on deep learning and knowledge graph according to claim 1, characterized in that, Obtain the preprocessed part image, specifically: The part images are sequentially filtered, cropped, scaled, and expanded to obtain preprocessed part images.

3. A parts category recognition system based on deep learning and knowledge graphs, characterized in that, The method described by any one of claims 1 to 2 includes: An image acquisition module is used to acquire the image of the part to be queried and the preprocessed image of the part. The model building module is used to establish a part image recognition model based on the VGG11 model, input the preprocessed part image into the part image recognition model, and obtain the trained image recognition model. The feature extraction module is used to extract features from the preprocessed part images using a trained image recognition model, and to obtain feature vectors contained in different part images; The image processing module is used to input the image of the part to be queried into the trained image recognition model to obtain feature vectors; The data processing module is used to process feature vectors and feature vectors contained in different part images to obtain detailed part information and realize part category recognition.

4. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the part category identification method based on deep learning and knowledge graph as described in any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the part category identification method based on deep learning and knowledge graph as described in any one of claims 1 to 2.

Images