Equipment identification method, device, computer equipment and storage medium

Abstract

The present invention relates to the field of computer vision technology, and provides an equipment recognition method, device, computer equipment and storage medium, the method comprising: obtaining an equipment image, predicting the equipment category of the equipment image based on a multi-layer label classification system, and obtaining a candidate label sequence; performing text detection on the equipment image, and determining the fine-grained label of the equipment image based on the extracted equipment digital model; constructing a graph structure corresponding to the multi-layer label classification system and with edge weights, the edge weights being the scores of each label in the candidate label sequence and the fine-grained label; and determining the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image. The use of this method can improve recognition speed and performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to an equipment recognition method, device, computer equipment and storage medium. Background Art

[0002] With the improvement of the information level of the army, the reconnaissance capability of the information system has been strengthened, which has led to a sharp increase in the reconnaissance images obtained by various platforms. Therefore, obtaining information from reconnaissance images has become a direct and effective means to accurately judge the battlefield situation. Among them, the research on equipment intelligent recognition and classification can quickly and effectively process reconnaissance images and provide important reference information.

[0003] Due to the wide variety of equipment and the small differences between many types of equipment, the detection and recognition of equipment images is more challenging than general image classification. There are currently three types of equipment recognition methods, including methods based on the same convolutional neural network, methods based on positioning-recognition, and methods based on network integration. However, most of these three methods consider equipment images in ideal scenarios, which reduces the speed and performance for equipment images in actual application scenarios. Summary of the invention

[0004] Based on this, it is necessary to provide an equipment identification method, device, computer equipment and storage medium that can improve the identification speed and performance in response to the above technical problems.

[0005] The present invention provides an equipment identification method, comprising:

[0006] Acquire an equipment image, predict the equipment category of the equipment image based on a multi-layer label classification system, and obtain a candidate label sequence;

[0007] Performing text detection on the equipment image, and determining a fine-grained label of the equipment image based on the extracted equipment digital model;

[0008] Constructing a graph structure corresponding to the multi-layer label classification system and having edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label;

[0009] Determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image

[0010] In one embodiment, the acquiring of the equipment image and predicting the equipment category of the equipment image based on a multi-layer label classification system to obtain a candidate label sequence includes:

[0011] A multi-layer label classification model is obtained based on a multi-layer label classification system and trained using a preset model strategy;

[0012] The multi-layer label classification model is used to predict the equipment category of the equipment image to obtain a candidate label sequence of the equipment image.

[0013] In one embodiment, the performing text detection on the equipment image and determining a fine-grained label of the equipment image based on the extracted equipment digital model includes:

[0014] Performing text detection on the equipment image using a text region detection model to determine a text region in the equipment image;

[0015] Performing OCR recognition on the text area to obtain the equipment digital model;

[0016] The fine-grained label of the equipment image is determined according to the mapping relationship between the equipment digital model and the fine-grained label.

[0017] In one embodiment, the constructing of a graph structure corresponding to the multi-layer label classification system and having edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label, includes:

[0018] Each label in the multi-layer label classification system is used as a node to construct a corresponding graph structure;

[0019] Determine the candidate tag sequence and the node corresponding to each tag in the fine-grained tag and the parent node corresponding to the node from the graph structure;

[0020] The score of the label is assigned as an edge weight to the directed edge between the node and the parent node.

[0021] In one embodiment, determining the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image includes:

[0022] Starting from the root node of the graph structure, traversing downwards, the weighted value of each traversal path is calculated according to the preset weight of each node and the weight of the edge;

[0023] The traversal path with the largest weighted value is determined, and the multi-layer label corresponding to the traversal path with the largest weighted value is used as the recognition result of the equipment image.

[0024] In one embodiment, before acquiring the equipment image, the method further includes:

[0025] Acquiring reconnaissance images;

[0026] The reconnaissance image is input into a VGG model, the non-equipment image in the reconnaissance image is filtered using the VGG model, and the equipment image is output.

[0027] In one embodiment, the weighted value is calculated as follows:

[0028]

[0029] Among them, P nm From the starting point S to the end point S nm There is a reachable path, z is the number of nodes on the path, S ni is one of the nodes on the path, W sni For node S ni The preset weight of score sni is the edge weight of the directed edge.

[0030] An equipment identification device, comprising:

[0031] A classification module, used to obtain an equipment image, predict the equipment category of the equipment image based on a multi-layer label classification system, and obtain a candidate label sequence;

[0032] A fine-grained recognition module, used to perform text detection on the equipment image and determine a fine-grained label of the equipment image based on the extracted equipment digital model;

[0033] A construction module, used to construct a graph structure corresponding to the multi-layer label classification system and with edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label;

[0034] The result determination module is used to determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image.

[0035] The present invention also provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above-mentioned equipment identification methods are implemented.

[0036] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned equipment identification methods.

[0037] The above-mentioned equipment recognition method, device, computer equipment and storage medium, after acquiring the equipment image, first predict the equipment category of the equipment image based on the multi-layer label classification system to obtain the candidate label sequence, and at the same time perform text detection on the equipment image, and determine the fine-grained label of the equipment image based on the extracted equipment digital model; then, by constructing a graph structure corresponding to the multi-layer label classification system and with edge weights, where the edge weights are the scores of each label in the candidate label sequence and the fine-grained label, finally determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image. This method divides the recognition of equipment into equipment system classification and fine-grained label recognition through a progressive strategy, and then uses the graph structure to combine the two results to determine the most accurate label as the recognition result, thereby further improving the recognition speed and performance of equipment images in actual application scenarios. In addition, when the overall structure of the equipment is similar and the difference between classes is small, by focusing on the key and distinguishable digital models, more favorable classification features can be extracted to improve the recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 FIG. 4 is a diagram of an application environment of an equipment identification method in an embodiment.

[0039] Figure 2 The figure is a flowchart of an equipment identification method in one embodiment.

[0040] Figure 3 A schematic diagram of a multi-layer label classification system configured in one embodiment.

[0041] Figure 4 A diagram showing the structure of a multi-layer label classification system in one embodiment.

[0042] Figure 5 A schematic diagram of a graph structure of a multi-layer label classification system with edge weights in one embodiment.

[0043] Figure 6 It is a structural block diagram of an equipment identification device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] The equipment identification method provided in this application can be applied to Figure 1In the application environment shown, the application environment involves a terminal 102 and a server 104. The terminal 102 communicates with the server 104 through a network. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0046] When the terminal 102 receives the recognition instruction, the above-mentioned equipment recognition method can be implemented by the terminal 102 alone. The terminal 102 can also send the recognition instruction to the communicating server 104, and the above-mentioned equipment recognition method can be implemented by the server 104. Taking the server 104 as an example, specifically, the server 104 obtains the equipment image, predicts the equipment category of the equipment image based on the multi-layer label classification system, and obtains the candidate label sequence; the server 104 performs text detection on the equipment image, and determines the fine-grained label of the equipment image based on the extracted equipment digital model; the server 104 constructs a graph structure corresponding to the multi-layer label classification system and with edge weights, and the edge weights are the scores of each label in the candidate label sequence and the fine-grained label; the server 104 determines the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image.

[0047] In one embodiment, Figure 2 As shown, a device identification method is provided, which is described by taking the method applied to a server as an example, and includes the following steps:

[0048] Step S201 : acquiring an equipment image, predicting the equipment category of the equipment image based on a multi-layer label classification system, and obtaining a candidate label sequence.

[0049] The equipment image refers to the image data of the equipment. The multi-layer label classification system is a label system with a hierarchical relationship set with the equipment classification as the label. This embodiment provides a schematic diagram of a multi-layer label classification system for equipment, such as Figure 3 shown. Figure 3 The multi-layer label classification system shown has 3 layers and a root node S. The first layer classification system contains three labels: naval equipment, army equipment, and air force equipment. The second layer classification system includes five labels: destroyer, landing ship, tank, fighter, and early warning aircraft. The third layer includes 11 labels. There is a subordinate relationship between the labels of the layers, and each label has only one parent label. The candidate label sequence is the classification result set obtained by classifying and identifying the equipment image.

[0050] Specifically, after receiving the recognition instruction, the server first obtains the equipment image to be recognized. Then, the server classifies the equipment image based on the pre-set multi-layer label classification system and determines the category of the equipment contained in the equipment image. The classification method can adopt any existing image classification or image recognition method. Finally, the server aggregates the obtained categories to obtain a candidate label sequence for the equipment image.

[0051] In one embodiment, step S201 includes: training a multi-layer label classification model based on a multi-layer label classification system and using a preset model strategy; predicting the equipment category of the equipment image using the multi-layer label classification model to obtain a candidate label sequence of the equipment image.

[0052] Specifically, the server trains the neural network model for classification prediction based on the set multi-layer label classification system, thereby obtaining a trained multi-layer label classification model. The training method of the multi-layer label classification model can be a conventional neural network training method, and the training strategy can be any one of a multi-model strategy (each model is responsible for multiple label classifications), a multi-task model strategy (a model supports multiple classification tasks, and each classification task is responsible for multiple label classifications), and a single model strategy (a model is responsible for all label classifications). There are differences in the speed and performance of classification of different model strategies, which can be selected according to the actual situation. The trained network structure can be any one of the existing ones, and the Xception model is preferably used in this embodiment. Then, the server inputs the equipment image into the trained multi-layer label classification model, and the multi-layer label classification model classifies and predicts the equipment image, outputs the candidate label set corresponding to the equipment image, and the server selects all the candidate labels in the candidate label set according to the probability score of the label and combines the candidate labels with higher probability into a candidate label sequence. For example, when the multi-layer label classification model predicts that the equipment image is naval equipment with a probability score of 0.98, a destroyer with a probability score of 0.92, a landing ship with a probability score of 0.87, a fighter with a probability score of 0.65, and an 18G electronic fighter with a probability score of 0.32, these five labels are candidate labels, and the candidate label sequence L is L = {(naval equipment, 0.98), (destroyer, 0.92), (landing ship, 0.87), (fighter, 0.65), (18G electronic fighter, 0.32)}. It should be understood that the multi-layer label classification model called in this embodiment can also be trained and deployed by the server in advance, and then the deployed multi-layer label classification model is directly called in this embodiment for classification prediction, so that the processing time spent on training can be saved during the recognition process.

[0053] Step S202 : Perform text detection on the equipment image and determine a fine-grained label of the equipment image based on the extracted equipment digital model.

[0054] Specifically, considering that the equipment images have extremely high similarity, the only way to distinguish them is the digital model of the equipment in the equipment image. Therefore, this implementation can improve the recognition accuracy by using the fine-grained recognition of the equipment digital model as the fine-grained label of the equipment image. That is, after obtaining the equipment image, the server extracts the text area in the equipment image by text detection, and further recognizes the characters in the text area to obtain the equipment digital model, and then determines the fine-grained label according to the equipment digital model.

[0055] In one embodiment, step S202 includes: performing text detection on the equipment image using a text region detection model to determine the text region in the equipment image; performing OCR recognition on the text region to obtain the equipment digital model; and determining the fine-grained label of the equipment image based on the mapping relationship between the equipment digital model and the fine-grained label.

[0056] Specifically, the text detection in this embodiment is preferably implemented using a text area detection model. The text area detection model can be trained using any existing neural network model. In this embodiment, DBNet is preferably used as the text area detection model. The server calls the trained text area detection model, inputs the equipment image into the text area detection model, and uses the text area detection model to detect and extract the text area in the equipment image. Then, the server uses OCR recognition to identify the text area, obtains the corresponding digital equipment model and the probability score of determining the digital model of the equipment, such as T = {115, 0.94}, where 115 is the digital model of the equipment, and 0.94 is the probability score of the digital model of the equipment being 115. Finally, the server determines the fine-grained label of the equipment image based on the mapping relationship between the digital model and the fine-grained label. For example, as shown in Table 1 below, the equipment with a digital model of 115 is the Rafael Bilada destroyer, and the fine-grained label is {Rafael Bilada destroyer, 0.94}.

[0057] Table 1 Mapping table of fine-grained labels and digital models

[0058] type Digital Model USS Barry 52 Rafael Bilarda destroyer 115 USS Milius 69 Tortuga landing ship 46 Germantown landing ship 42 M1A2 Main Battle Tank M1A2 Leopard 2 Main Battle Tank none 18E Super Hornet 18E 18F Hornet fighter 18F 8G Electronic Fighter 8G E-2C Hawkeye early warning aircraft none

[0059] Step S203, constructing a graph structure corresponding to the multi-layer label classification system and having edge weights, where the edge weights are the scores of each label in the candidate label sequence and the fine-grained label.

[0060] Specifically, a multi-layer label classification system is established with a graph or tree structure, and the label scores of each label in the candidate label sequence and the fine-grained labels are used as the weights of the edges of the graph structure or tree structure to obtain a tree structure with edge weights.

[0061] In one embodiment, step S203 includes: constructing a corresponding graph structure using each label in the multi-layer label classification system as a node; determining the nodes corresponding to each label in the candidate label sequence and the fine-grained label and the parent node corresponding to the node from the graph structure; and assigning the label score as an edge weight to the directed edge between the node and the parent node.

[0062] Specifically, we first use the graph structure to Figure 3 The multi-layer label classification system shown in Figure 1 is modeled and the following is obtained: Figure 4 The graph structure corresponding to the multi-layer label classification system is shown. Among them, each label in the multi-layer label classification system is regarded as each node in the graph structure, and a directed edge is established between each node and its parent node. The direction of the edge is from the parent node to the child node, and the initial weight of the directed edge is 0 at this time. Then, the nodes corresponding to each candidate label in the candidate label sequence and the corresponding parent nodes are found in the tree structure, and the probability score corresponding to the candidate label is used as the weight of the directed edge between the node and its parent node. Similarly, the probability score of the fine-grained label is assigned to the corresponding directed edge in the same way, thereby obtaining a graph structure with edge weights, such as Figure 5 shown.

[0063] Step S204: determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image.

[0064] Specifically, for each path in the graph structure, its weighted value is calculated, and the path with the largest weighted value is selected as the weighted maximum path, and then the multi-layer label corresponding to the weighted maximum path is determined as the recognition result of the equipment image.

[0065] In one embodiment, step S204 includes: traversing downward from the root node of the graph structure as the starting point, calculating the weighted value of each traversal path according to the preset weights of each node and the weights of the edges; determining the traversal path with the largest weighted value, and using the multi-layer label corresponding to the traversal path with the largest weighted value as the recognition result of the equipment image.

[0066] Specifically, in the graph structure, starting from the root node S, traverse down to the last layer, such as Figure 4-5 As shown, the traversal reaches the third layer, and each label node in the third layer {USS Berry, USS Rafael Bilarda, etc. E-2C Hawkeye early warning aircraft} is used as the final point. For each traversed path, the Dijkstra shortest path algorithm is used to calculate the weighted value according to the following formula:

[0067]

[0068] Among them, P nm From the starting point S to the end point S nmThere is a reachable path, z is the number of nodes on the path, S ni is one of the nodes on the path, W sni For node S ni The weight of each node is preferably 1. sni is the edge weight of the directed edge. For example, the path of the USS Barry destroyer node is: root node S->naval equipment->destroyer->USS Barry destroyer, and the path score is 0.633. Then, the server selects the maximum path based on the weighted value. Figure 5 Take the Rafael Biralda destroyer node path as an example, where the path nodes include: root node, naval equipment, destroyer, Rafael Biralda destroyer, and the path score is 0.946. Delete the nodes with a weight of 0 on the path, and restore the remaining nodes according to the multi-layer label classification system to obtain the equipment image multi-layer label recognition result: the first layer label is: naval equipment, the second layer label is: destroyer, and the third layer label is: Rafael Biralda destroyer.

[0069] After acquiring the equipment image, the above equipment recognition method first predicts the equipment category of the equipment image based on the multi-layer label classification system to obtain the candidate label sequence, and at the same time performs text detection on the equipment image, and determines the fine-grained label of the equipment image based on the extracted equipment digital model; then, a graph structure corresponding to the multi-layer label classification system and with edge weights is constructed, where the edge weights are the scores of each label in the candidate label sequence and the fine-grained label, and finally determines the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image. This method divides the equipment recognition into equipment system classification and fine-grained label recognition through a progressive strategy, and then uses the graph structure to combine the two results to determine the most accurate label as the recognition result, thereby further improving the recognition speed and performance of the equipment image in the actual application scenario. In addition, when the overall structure of the equipment is similar and the difference between classes is small, by focusing on the key separable digital model, more favorable classification features can be extracted to improve the recognition performance.

[0070] In one embodiment, before step S201, the method further includes: acquiring a reconnaissance image; inputting the reconnaissance image into a VGG model, filtering non-equipment images in the reconnaissance image using the VGG model, and outputting an equipment image.

[0071] Specifically, since most of the massive images on the Internet are images that users are not interested in, directly using them for equipment image recognition will inevitably lead to a decrease in overall performance. Therefore, this embodiment designs a filtering module before acquiring the equipment image, and uses the lightweight classification model VGG to filter the acquired reconnaissance image, and obtains the equipment image containing the equipment by filtering the VGG model. Among them, the VGG model training data can be collected equipment images and non-equipment images, and the training method can adopt a fine-tuning strategy. In this embodiment, by pre-screening and filtering images, it is ensured that the images entering the equipment image recognition process are all images containing equipment, thereby improving the recognition performance.

[0072] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0073] In one embodiment, Figure 6 As shown, a device for identifying equipment is provided, comprising:

[0074] The classification module 601 is used to obtain an equipment image, predict the equipment category of the equipment image based on a multi-layer label classification system, and obtain a candidate label sequence;

[0075] A fine-grained recognition module 602 is used to perform text detection on the equipment image and determine a fine-grained label of the equipment image based on the extracted equipment digital model;

[0076] A construction module 603 is used to construct a graph structure corresponding to the multi-layer label classification system and with edge weights, where the edge weights are the scores of each label in the candidate label sequence and the fine-grained label;

[0077] The result determination module 604 is used to determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image.

[0078] In one embodiment, the classification module 601 is further used to obtain a multi-layer label classification model based on a multi-layer label classification system and trained using a preset model strategy; and to predict the equipment category of the equipment image using the multi-layer label classification model to obtain a candidate label sequence of the equipment image.

[0079] In one embodiment, the fine-grained recognition module 602 is also used to perform text detection on the equipment image using a text region detection model to determine the text region in the equipment image; perform OCR recognition on the text region to obtain the equipment digital model; and determine the fine-grained label of the equipment image based on the mapping relationship between the equipment digital model and the fine-grained label.

[0080] In one embodiment, the construction module 603 is also used to construct a corresponding graph structure using each label in the multi-layer label classification system as a node; determine the nodes corresponding to each label in the candidate label sequence and the fine-grained label and the parent node corresponding to the node from the graph structure; and assign the label score as the edge weight to the directed edge between the node and the parent node.

[0081] In one embodiment, the result determination module 604 is also used to traverse downward from the root node of the graph structure as the starting point, calculate the weighted value of each traversal path according to the preset weight of each node and the weight of the edge; determine the traversal path with the largest weighted value, and use the multi-layer label corresponding to the traversal path with the largest weighted value as the recognition result of the equipment image.

[0082] In one embodiment, a filtering module is further included, which is used to obtain a reconnaissance image; input the reconnaissance image into a VGG model, use the VGG model to filter non-equipment images in the reconnaissance image, and output an equipment image.

[0083] For the specific definition of the equipment identification device, please refer to the definition of the equipment identification method above, which will not be repeated here. Each module in the above-mentioned equipment identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned equipment identification method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0084] In one embodiment, a computer device is provided, which may be a server, including a processor, a memory and a network interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying equipment is implemented. Exemplarily, the computer program can be divided into one or more modules, one or more modules are stored in the memory, and are executed by the processor to complete the present invention. One or more modules can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0085] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0086] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0087] Those skilled in the art will appreciate that the computer device structure shown in this embodiment is merely a partial structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the present invention is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0088] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0089] Obtain equipment images, predict equipment categories of equipment images based on a multi-layer label classification system, and obtain candidate label sequences;

[0090] Perform text detection on equipment images and determine fine-grained labels of equipment images based on the extracted equipment digital models;

[0091] Construct a graph structure with edge weights corresponding to the multi-layer label classification system. The edge weights are the scores of each label in the candidate label sequence and fine-grained labels.

[0092] The multi-layer labels corresponding to the weighted maximum path in the graph structure are determined as the recognition results of the equipment image.

[0093] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the multi-layer label classification system and using a preset model strategy to train a multi-layer label classification model; using the multi-layer label classification model to predict the equipment category of the equipment image, and obtain a candidate label sequence for the equipment image.

[0094] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using a text area detection model to perform text detection on the equipment image to determine the text area in the equipment image; performing OCR recognition on the text area to obtain the equipment digital model; and determining the fine-grained label of the equipment image based on the mapping relationship between the equipment digital model and the fine-grained label.

[0095] In one embodiment, when the processor executes the computer program, the following steps are also implemented: constructing a corresponding graph structure using each label in the multi-layer label classification system as a node; determining the nodes corresponding to each label in the candidate label sequence and the fine-grained label and the parent node corresponding to the node from the graph structure; and assigning the label score as the edge weight to the directed edge between the node and the parent node.

[0096] In one embodiment, when the processor executes the computer program, the following steps are also implemented: traversing downward from the root node of the graph structure as the starting point, calculating the weighted value of each traversal path according to the preset weights of each node and the weights of the edges; determining the traversal path with the largest weighted value, and using the multi-layer label corresponding to the traversal path with the largest weighted value as the recognition result of the equipment image.

[0097] In one embodiment, when the processor executes the computer program, the following steps are also implemented: acquiring a reconnaissance image; inputting the reconnaissance image into a VGG model, using the VGG model to filter non-equipment images in the reconnaissance image, and outputting an equipment image.

[0098] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0099] Obtain equipment images, predict equipment categories of equipment images based on a multi-layer label classification system, and obtain candidate label sequences;

[0100] Perform text detection on equipment images and determine fine-grained labels of equipment images based on the extracted equipment digital models;

[0101] Construct a graph structure with edge weights corresponding to the multi-layer label classification system. The edge weights are the scores of each label in the candidate label sequence and fine-grained labels.

[0102] The multi-layer labels corresponding to the weighted maximum path in the graph structure are determined as the recognition results of the equipment image.

[0103] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: a multi-layer label classification model is obtained by training based on a multi-layer label classification system and using a preset model strategy; the equipment category of the equipment image is predicted using the multi-layer label classification model to obtain a candidate label sequence for the equipment image.

[0104] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: using a text region detection model to perform text detection on the equipment image to determine the text region in the equipment image; performing OCR recognition on the text region to obtain the equipment digital model; and determining the fine-grained label of the equipment image based on the mapping relationship between the equipment digital model and the fine-grained label.

[0105] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: constructing a corresponding graph structure using each label in the multi-layer label classification system as a node; determining the nodes corresponding to each label in the candidate label sequence and the fine-grained label and the parent nodes corresponding to the nodes from the graph structure; and assigning the label score as an edge weight to the directed edge between the node and the parent node.

[0106] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: starting from the root node of the graph structure, traversing downward, calculating the weighted value of each traversal path according to the preset weights of each node and the weights of the edges; determining the traversal path with the largest weighted value, and using the multi-layer label corresponding to the traversal path with the largest weighted value as the recognition result of the equipment image.

[0107] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a reconnaissance image; inputting the reconnaissance image into a VGG model, using the VGG model to filter non-equipment images in the reconnaissance image, and outputting an equipment image.

[0108] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0109] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for equipment identification, characterized in that: include: Acquire an equipment image, predict the equipment category of the equipment image based on a multi-layer label classification system, and obtain a candidate label sequence; Performing text detection on the equipment image, and determining a fine-grained label of the equipment image based on the extracted equipment digital model; Constructing a graph structure corresponding to the multi-layer label classification system and having edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label; The multi-layer labels corresponding to the weighted maximum path in the graph structure are determined as the recognition result of the equipment image.

2. The method according to claim 1, characterized in that The acquiring of the equipment image and predicting the equipment category of the equipment image based on a multi-layer label classification system to obtain a candidate label sequence includes: A multi-layer label classification model is obtained based on a multi-layer label classification system and trained using a preset model strategy; The multi-layer label classification model is used to predict the equipment category of the equipment image to obtain a candidate label sequence of the equipment image.

3. The method according to claim 1, characterized in that The performing text detection on the equipment image and determining a fine-grained label of the equipment image based on the extracted equipment digital model includes: Performing text detection on the equipment image using a text region detection model to determine a text region in the equipment image; Performing OCR recognition on the text area to obtain the equipment digital model; The fine-grained label of the equipment image is determined according to the mapping relationship between the equipment digital model and the fine-grained label.

4. The method according to claim 1, characterized in that: The constructing a graph structure corresponding to the multi-layer label classification system and having edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label, includes: Each label in the multi-layer label classification system is used as a node to construct a corresponding graph structure; Determine the candidate tag sequence and the node corresponding to each tag in the fine-grained tag and the parent node corresponding to the node from the graph structure; The score of the label is assigned as an edge weight to the directed edge between the node and the parent node.

5. The method according to claim 1, characterized in that The step of determining the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image includes: Starting from the root node of the graph structure, traversing downwards, the weighted value of each traversal path is calculated according to the preset weight of each node and the weight of the edge; The traversal path with the largest weighted value is determined, and the multi-layer label corresponding to the traversal path with the largest weighted value is used as the recognition result of the equipment image.

6. The method according to claim 5, characterized in that The calculation formula of the weighted value is as follows: Among them, P nm From the starting point S to the end point S nm There is a reachable path, z is the number of nodes on the path, S ni is one of the nodes on the path, W sni For node S ni The preset weight of score sni is the edge weight of the directed edge.

7. The method according to claim 1, characterized in that Before acquiring the equipment image, the method further includes: Acquiring reconnaissance images; The reconnaissance image is input into a VGG model, the non-equipment image in the reconnaissance image is filtered using the VGG model, and the equipment image is output.

8. An equipment identification device, characterized in that: include: A classification module, used to obtain an equipment image, predict the equipment category of the equipment image based on a multi-layer label classification system, and obtain a candidate label sequence; A fine-grained recognition module, used to perform text detection on the equipment image and determine a fine-grained label of the equipment image based on the extracted equipment digital model; A construction module, used to construct a graph structure corresponding to the multi-layer label classification system and with edge weights, wherein the edge weights are scores of each label in the candidate label sequence and the fine-grained label; The result determination module is used to determine the multi-layer label corresponding to the weighted maximum path in the graph structure as the recognition result of the equipment image.

9. A computer device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: The processor is used to implement the equipment identification method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the equipment identification method according to any one of claims 1 to 7 is implemented.

Images