A high-efficiency retrieval method for ore-dressing equipment accessory images based on deep hash learning

Abstract

The application relates to the field of artificial intelligence and intelligent information retrieval technology, and discloses a high-efficiency retrieval method for images of ore-dressing equipment accessories based on deep hash learning. The method comprises the following steps: constructing a differentiable MinHash layer based on MinHash and differentiable operation, combining a Transformer backbone network and a hash layer to obtain an image representation model; obtaining the images of industrial accessories, inputting the images into the image representation model, and obtaining the minimum hash code of the industrial accessory images through discretization; combining structured data to construct a bidirectional index; obtaining the bidirectional index of all industrial accessory images and establishing an accessory feature database; obtaining the images of the industrial accessories to be queried, inputting the images into the image representation model, obtaining the corresponding minimum hash code through discretization, obtaining the corresponding structured data based on the minimum hash code, finding the accessories based on the structured data, and completing the replacement work, so that the problem that existing image learning retrieval methods cannot accurately and efficiently identify accessories is solved.

Description

Technical Field

[0001] This invention relates to the field of industrial artificial intelligence and intelligent information retrieval technology, and in particular to an end-to-end minimum hash image representation algorithm. Background Technology

[0002] In the non-ferrous metal ore beneficiation process, equipment spare parts management has long faced challenges such as a wide variety of spare parts, unclear labeling of old equipment, and frequent upgrades. Taking a large copper mine beneficiation plant as an example, its production line involves over a million types of spare parts (including materials), of which approximately 30% are old materials with worn nameplates or unclear model markings. Especially for small spare parts such as steel rings and screws, nameplate information is often difficult to identify due to long-term use or oil contamination, making accurate retrieval in the WMS (Warehouse Management System) impossible using textual information such as name, model, and specifications. Furthermore, the proportion of newly added replacement spare parts reaches as high as 15% annually, further exacerbating the complexity of spare parts identification and management.

[0003] The existing technology has the following obvious drawbacks:

[0004] First, for accessories with worn, contaminated, or completely missing nameplates, existing deep learning retrieval methods based on image content lack accurate image content recognition means and cannot accurately identify and match based on image visual features, resulting in extremely low retrieval efficiency or even complete failure for such accessories.

[0005] Secondly, faced with the massive data environment brought about by the wide variety, large quantity, and frequent updates of spare parts, existing deep learning retrieval methods based on image content are slow to respond and difficult to achieve fast and efficient spare parts retrieval and matching, which seriously restricts the efficiency of equipment maintenance and inventory management.

[0006] Therefore, there is an urgent need for a rapid and accurate retrieval method and system for mineral processing equipment parts based on image recognition and intelligent retrieval technology that can overcome the above-mentioned defects, so as to significantly improve the intelligence level, response speed and emergency handling capabilities of parts management. Summary of the Invention

[0007] This invention provides an efficient image retrieval method for mineral processing equipment accessories based on deep hash learning, in order to solve the problem that existing image learning retrieval methods are unable to accurately and efficiently identify accessories.

[0008] To achieve the above objectives, the present invention employs the following technical solution:

[0009] This invention provides an efficient image retrieval method for mineral processing equipment accessories based on deep hash learning, comprising:

[0010] Step 1: Construct a differentiable MinHash layer based on MinHash and differentiable operations, and combine the already constructed Transformer backbone network and hash layer to obtain the image representation model;

[0011] Step 2: Obtain the original image of a single industrial part and input the original image into the image representation model. Extract image features through the Transformer backbone network. The hash layer obtains continuous hash codes based on the image features. The differentiable MinHash layer obtains continuous minimum hash signature vectors based on the continuous hash codes. Discretize the continuous minimum hash signature vectors to obtain the minimum hash code corresponding to the industrial part. Then obtain the structured data corresponding to a single industrial part and construct a bidirectional index with the structured data and the minimum hash code.

[0012] Step 3: Repeat step 2 to obtain the bidirectional index of each industrial part, and build a part feature database based on the bidirectional index of all industrial parts;

[0013] Step 4: Obtain the original image of the industrial part to be queried. Input the original image of the industrial part to be queried into the image representation model, obtain the minimum hash code corresponding to the industrial part to be queried, query the part feature database based on the minimum hash code, obtain the structured data of the industrial part to be queried, obtain new industrial parts based on the structured data of the industrial part to be queried, and replace them.

[0014] Furthermore, the construction of a differentiable MinHash layer based on MinHash and differentiable operations includes: constructing a differentiable MinHash layer based on a learnable parameterized hash function and defining its predetermined batch and approximate calculation of the differentiable minimum value;

[0015] The approximate calculation of the differentiable minimum value is expressed by the following formula:

[0016] ;

[0017] in, Indicates the current consecutive hash code; Represents consecutive hash codes The Minimum estimate of each signature; Represents the defined mathematical expectation; This represents the sampling distribution corresponding to the mathematical expectation; Indicates the first A hash function, , The number of maximum hash functions;

[0018] The sampling distribution of the mathematical expectation is represented by the following formula:

[0019] ;

[0020] in, This indicates the degree of control over discretization; No. The consecutive hash codes of the images, ; Indicates the pre-order batch. , and These represent the consecutive hash codes of the 1st and Nth images, respectively.

[0021] Through the above design, unlike the traditional MinHash method which relies on a fixed random hash function, the learnable parameterized hash function is constructed based on linear projection and modular arithmetic, and its weights and biases can be automatically optimized through training. This learnable hashing mechanism not only retains the theoretical advantages of MinHash in similarity retrieval, but also can adaptively adjust according to the actual task data distribution, greatly improving the discriminativeness and adaptability of the hash mapping, thereby enhancing the method's performance in diverse image data scenarios.

[0022] Furthermore, the step of combining the pre-constructed Transformer backbone network and hash layer to obtain the image representation model includes: constructing the image representation model according to the network structure of Transformer backbone network, hash layer, and differentiable MinHash layer;

[0023] The differentiable MinHash layer converts consecutive hash codes into consecutive minimum hash signature vectors during forward propagation;

[0024] The gradient calculation for a differentiable MinHash layer during backpropagation is expressed by the following formula:

[0025] ;

[0026] in, To express differentiation; Indicates trainable weights; Represents the consecutive hash codes of the candidates.

[0027] Furthermore, in step 2, the minimum hash code is obtained through the following steps:

[0028] Step 201: Obtain the original image of a single industrial part, input the original image into the image representation model to obtain the corresponding image depth features, and then obtain the continuous hash code corresponding to the image depth features through the hash layer;

[0029] Step 202: The differentiable MinHash layer performs differentiable operations and temperature annealing mechanisms in a predetermined batch to convert continuous hash codes into continuous minimum hash signature vectors;

[0030] The temperature annealing mechanism is expressed by the following formula:

[0031] ;

[0032] in, express The temperature corresponding to the number of training steps; Indicates the initial temperature; Indicates the attenuation coefficient; Indicates the number of training steps;

[0033] Step 203: Discretize the continuous minimum hash signature vectors in the inference stage of the image representation model to obtain the minimum hash code corresponding to the industrial parts.

[0034] Through the above design, combined with a temperature annealing mechanism, the discrete signature is gradually approximated. This design allows the gradient to backpropagate from the final retrieval or representation loss to the feature extraction network, realizing the integrated optimization of deep feature learning and hash signature generation. It breaks through the technical bottleneck of "separation of feature extraction and hashing" in existing methods and has significant theoretical innovation.

[0035] Furthermore, the step of discretizing the continuous minimum hash signature vectors to obtain the minimum hash code corresponding to the industrial parts during the inference stage of the image representation model includes: rounding the continuous minimum hash signature vectors to the nearest integer to obtain the minimum hash code during the inference stage of the image representation model.

[0036] The rounding is expressed by the following formula:

[0037] ;

[0038] in, Represented as the floor function; This represents the consecutive minimum hash signature vector obtained in the previous step; Represents the smallest hash code after discretization; Represents the set of integers; Represents the set of real numbers;

[0039] Furthermore, the image representation model uses semantic similarity mean square loss and discrete loss function to quantify the loss during the discretization process of continuous minimum hash signature vectors;

[0040] The discrete loss function is expressed by the following formula:

[0041] ;

[0042] in, Represented as the Frobenius norm; The rounding function represents the rounding to the nearest integer.

[0043] Furthermore, the discrete loss function is processed using a Straight-Through Estimator during gradient calculation. Non-differentiable problem.

[0044] Furthermore, the structured data corresponding to the industrial parts includes the part name, model specifications, manufacturer, material code, purchase price, and installation instructions.

[0045] Furthermore, the query in the parts feature database based on the minimum hash code includes: matching all minimum hash codes stored in the parts feature database based on the minimum hash code combined with cosine similarity and hash collision detection mechanism.

[0046] Beneficial effects:

[0047] This invention seamlessly embeds the differentiable MinHash mechanism into the representation learning framework of deep neural networks, constructing a complete learnable pipeline from feature extraction and continuous hash code generation to differentiable MinHash computation, ultimately outputting the minimum hash code (discrete signature vector) through discretization. This integrated scheme achieves differentiable optimization of the hash mapping during the training phase, and only requires simple discretization operations to complete efficient retrieval during the inference phase. It avoids the complex process of additional post-processing or transformation of features in traditional methods, ensuring the system's uniformity and end-to-end characteristics, and significantly improving the efficiency and accuracy in large-scale image retrieval and similarity matching tasks.

[0048] This invention provides an efficient image retrieval method for mineral processing equipment accessories based on deep hash learning. It designs a differentiable hash mapping framework, successfully solving the non-differentiability problem caused by the discreteness of traditional minimum hash methods. This method achieves end-to-end training from the original image space to the hash space, enabling the generated image representation to combine the strong representational power of deep visual features with the efficient retrieval advantages of locality-sensitive hashing. This provides a new solution for fast approximate nearest neighbor search of high-dimensional data. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an efficient image retrieval method for mineral processing equipment accessories based on deep hash learning, according to an embodiment of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0052] Please see Figure 1 This application provides an efficient image retrieval method for mineral processing equipment accessories based on deep hash learning, comprising the following steps:

[0053] Step 1: Construct a differentiable MinHash layer based on MinHash and differentiable operations, and combine the already constructed Transformer backbone network and hash layer to obtain the image representation model;

[0054] Regarding the differentiable MinHash layer, the differentiable MinHash layer is based on a learnable parameterized hash function and is constructed by defining a predetermined batch and approximating the differentiable minimum value.

[0055] Specifically, the input features are first mapped to multiple consecutive hash codes using a learnable parameterized hash function, as expressed by the following formula:

[0056] ;

[0057] in, Indicates the first ( , (This represents the maximum number of hash functions) (hash functions); Indicates the current consecutive hash code; These are trainable weights, vectors It follows a multivariate Gaussian distribution, and its initial value is from Mid-sampling, express The identity matrix, Indicates the dimension of consecutive hash codes; It is a trainable bias, scalar It follows a uniform distribution, and its initial value is from Mid-sampling, Represents a fixed prime number; Represents modulo operation; Indicates the inner product;

[0058] The approximate calculation of the differentiable minimum value is expressed by the following formula:

[0059] ;

[0060] in, Indicates the current consecutive hash code; Represents consecutive hash codes The Minimum estimate of each signature; Represents the defined mathematical expectation; This represents the sampling distribution corresponding to the mathematical expectation; Indicates the first A hash function, , The number of maximum hash functions;

[0061] The sampling distribution of the mathematical expectation is represented by the following formula:

[0062] ;

[0063] in, This indicates the degree of control over discretization; No. The consecutive hash codes of the images, ; Indicates the pre-order batch. , and These represent the consecutive hash codes of the 1st and Nth images, respectively.

[0064] After constructing the differentiable MinHash layer, an image representation model is built according to the network structure of Transformer backbone network, hash layer, and differentiable MinHash layer. Then, during the forward propagation process, the differentiable MinHash layer converts continuous hash codes into continuous minimum hash signature vectors.

[0065] During backpropagation, the corresponding gradient is calculated using the following formula:

[0066] ;

[0067] in, To express differentiation; Indicates trainable weights; Represents the consecutive hash codes of the candidates.

[0068] During the training process of the image representation model, high-definition camera equipment is used to acquire multi-angle images of various industrial parts and to take pictures under different lighting conditions in order to obtain rich and three-dimensional visual information, enhance the diversity of image data and the representation ability of the image representation model. The images of various industrial parts also need to be preprocessed.

[0069] Here, open the image handle and generate... object, then to The object performs image channel scaling, normalization, and reconstructs data distribution. The object size is adjusted, and after a series of preprocessing steps, the corresponding high-dimensional feature vector is obtained. Then, the high-dimensional feature vector is input into the image representation model for training.

[0070] Regarding image channel specifications, the specifications are (height, width, channels). Data type transformation to (channel, height, width) Data type;

[0071] Normalization involves dividing the values ​​of each dimension (height, width, and channel) by 255 to normalize the values ​​to [0,1].

[0072] Reconstruct the data distribution. Given an input mean of [0.485, 0.456, 0.406] and a standard deviation of [0.229, 0.224, 0.225], reconstruct the numerical distribution to the interval (-1, 1).

[0073] Adjusting object size The object is a standard 224×224 size;

[0074] During training, the image representation model uses semantic similarity mean square loss and discrete loss function to construct a loss function, which is used to quantify the loss of the discretization process of the differentiable MinHash layer.

[0075] Then the loss function Represented as:

[0076] ;

[0077] in, This represents the weighting coefficient, which is an empirical value and is set to 0.5 in this embodiment.

[0078] The semantic approximation mean square loss is expressed by the following formula:

[0079] ;

[0080] in, Mean squared loss represents semantic approximation; and Both represent high-dimensional feature vectors; and Both represent consecutive minimum hash signature vectors; This represents the cosine similarity.

[0081] The discrete loss function is expressed by the following formula:

[0082] ;

[0083] in, Represented as the Frobenius norm; This represents the consecutive minimum hash signature vector obtained in the previous step; Indicates the largest number; Indicates the number of maximum hash functions; This represents the rounding function. The discrete loss function is processed using a Straight-ThroughEstimator during gradient calculation. Non-differentiable problem.

[0084] The learnable parameters in the model are optimized by differentiating the loss function and using gradient backpropagation. During training, the Adam optimizer is used to adaptively adjust the learning rate to ensure stable convergence and accelerate the training process. After completing a predetermined number of iterations, the model parameters with the lowest loss on the validation set are selected as the final optimal image representation model for subsequent feature extraction and retrieval.

[0085] Step 2: Obtain the original image of a single industrial part and input the original image into the image representation model. Extract image features through the Transformer backbone network. The hash layer obtains continuous hash codes based on the image features. The differentiable MinHash layer obtains continuous minimum hash signature vectors based on the continuous hash codes. Discretize the continuous minimum hash signature vectors to obtain the minimum hash code corresponding to the industrial part. Then obtain the structured data corresponding to a single industrial part and construct a bidirectional index with the structured data and the minimum hash code.

[0086] Step 201: Obtain the original image of a single industrial part, input the original image into the image representation model to obtain the corresponding image depth features, and then obtain the continuous hash code corresponding to the image depth features through the hash layer;

[0087] Step 202: The differentiable MinHash layer performs differentiable operations and temperature annealing mechanisms in a predetermined batch to convert continuous hash codes into continuous minimum hash signature vectors;

[0088] The approximate calculation of the differentiable minimum value is expressed by the following formula:

[0089] ;

[0090] in, Indicates the current consecutive hash code; Represents consecutive hash codes The Minimum estimate of each signature; Represents the defined mathematical expectation; This represents the sampling distribution corresponding to the mathematical expectation; Indicates the first A hash function, , The number of maximum hash functions;

[0091] The sampling distribution of the mathematical expectation is represented by the following formula:

[0092] ;

[0093] in, This indicates the degree of control over discretization; No. The consecutive hash codes of the images, ; Indicates the pre-order batch. , and These represent the consecutive hash codes of the 1st and Nth images, respectively.

[0094] The temperature annealing mechanism is represented by the following formula:

[0095] ;

[0096] in, express The temperature corresponding to the number of training steps; Indicates the initial temperature; Indicates the attenuation coefficient; This indicates the number of training steps.

[0097] Step 203: Discretize the continuous minimum hash signature vectors in the inference stage of the image representation model to obtain the minimum hash code corresponding to the industrial parts;

[0098] Rounding to the nearest integer in a discrete process is expressed by the following formula:

[0099] ;

[0100] in, Represented as the floor function; This represents the consecutive minimum hash signature vector obtained in the previous step; Represents the smallest hash code after discretization; Represents the set of integers; Represents the set of real numbers;

[0101] Step 204: Obtain the structured data corresponding to the industrial part, including part name, model specifications, manufacturer, material code, purchase price and installation instructions. Establish a bidirectional index between the structured data of the industrial part and the minimum hash code, and save the bidirectional index for the construction of the part feature database in Step 3.

[0102] Step 3: Repeat step 2 to obtain the bidirectional index of each industrial part, and build a part feature database based on the bidirectional index of all industrial parts.

[0103] Repeat step 2 for each industrial part in the current inventory to obtain the minimum hash code for each industrial part, and then obtain the structured data for each industrial part. Create a bidirectional index between the structured data and the minimum hash code, and build a part feature database from all the bidirectional indexes.

[0104] Step 4: Obtain the original image of the industrial part to be queried. Input the original image of the industrial part to be queried into the image representation model, obtain the minimum hash code corresponding to the industrial part to be queried, query the part feature database based on the minimum hash code, obtain the structured data of the industrial part to be queried, obtain new industrial parts based on the structured data of the industrial part to be queried, and replace them.

[0105] During on-site operation, when equipment malfunctions and an unknown industrial part needs to be replaced, operators can use a mobile terminal (such as an explosion-proof mobile phone or industrial tablet) to capture a real-time image of the industrial part and upload it to the system. Upon receiving the image, the system immediately invokes the deployed image representation model to extract the minimum hash code of the query image within seconds. Subsequently, the retrieval engine, based on a cosine similarity calculation strategy, performs rapid matching and sorting in the part feature database. Utilizing hash collision detection and fault tolerance mechanisms, it effectively addresses on-site interference factors such as complex image backgrounds, partial occlusion, or inconsistent lighting, accurately returning the most similar part result. The returned results not only include all technical parameters, inventory status, and storage location of the matched part, but also display practical information such as installation instructions, procurement links, and recommendations for similar alternative parts, greatly improving maintenance efficiency and spare parts management, and reducing equipment downtime.

[0106] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for efficient image retrieval of mineral processing equipment accessories based on deep hash learning, characterized in that, Includes the following steps: Step 1: Construct a differentiable MinHash layer based on MinHash and differentiable operations, and combine the already constructed Transformer backbone network and hash layer to obtain the image representation model; The construction of a differentiable MinHash layer based on MinHash and differentiable operations includes: constructing a differentiable MinHash layer based on a learnable parameterized hash function, defining its predetermined batch and approximate calculation of the differentiable minimum value; The approximate calculation of the differentiable minimum value is expressed by the following formula: ； in, Indicates the current consecutive hash code; Represents consecutive hash codes The Minimum estimate of each signature; Represents the defined mathematical expectation; This represents the sampling distribution corresponding to the mathematical expectation; Indicates the first A hash function, , The number of maximum hash functions; The sampling distribution of the mathematical expectation is represented by the following formula: ； in, This indicates the degree of control over discretization; No. The consecutive hash codes of the images, ; Indicates the pre-order batch. , and These represent the consecutive hash codes of the 1st and Nth images, respectively; Step 2: Obtain the original image of a single industrial part and input the original image into the image representation model. Extract image features through the Transformer backbone network. The hash layer obtains continuous hash codes based on the image features. The differentiable MinHash layer obtains continuous minimum hash signature vectors based on the continuous hash codes. Discretize the continuous minimum hash signature vectors to obtain the minimum hash code corresponding to the industrial part. Then obtain the structured data corresponding to a single industrial part and construct a bidirectional index with the structured data and the minimum hash code. The minimum hash code is obtained through the following steps: Step 201: Obtain the original image of a single industrial part, input the original image into the image representation model to obtain the corresponding image depth features, and then obtain the continuous hash code corresponding to the image depth features through the hash layer; Step 202: The differentiable MinHash layer performs differentiable operations and temperature annealing mechanisms in a predetermined batch to convert continuous hash codes into continuous minimum hash signature vectors; Step 203: Discretize the continuous minimum hash signature vectors in the inference stage of the image representation model to obtain the minimum hash code corresponding to the industrial parts; Step 3: Repeat step 2 to obtain the bidirectional index of each industrial part, and build a part feature database based on the bidirectional index of all industrial parts; Step 4: Obtain the original image of the industrial part to be queried. Input the original image of the industrial part to be queried into the image representation model, obtain the minimum hash code corresponding to the industrial part to be queried, query the part feature database based on the minimum hash code, obtain the structured data of the industrial part to be queried, obtain new industrial parts based on the structured data of the industrial part to be queried, and replace them.

2. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 1, characterized in that, The method of combining the pre-constructed Transformer backbone network and hash layer to obtain the image representation model includes: constructing the image representation model according to the network structure of Transformer backbone network, hash layer, and differentiable MinHash layer; The differentiable MinHash layer converts consecutive hash codes into consecutive minimum hash signature vectors during forward propagation; The gradient calculation for a differentiable MinHash layer during backpropagation is expressed by the following formula: ； in, To express differentiation; Indicates trainable weights; Represents the consecutive hash codes of the candidates.

3. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 1, characterized in that, The step of discretizing the continuous minimum hash signature vectors to obtain the minimum hash code corresponding to the industrial parts in the inference stage of the image representation model includes: rounding the continuous minimum hash signature vectors to the nearest integer to obtain the minimum hash code in the inference stage of the image representation model. The rounding is expressed by the following formula: ； in, Represented as the floor function; This represents the consecutive minimum hash signature vector obtained in the previous step; Represents the smallest hash code after discretization; Represents the set of integers; It represents the set of real numbers.

4. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 3, characterized in that, The image representation model uses semantic similarity mean square loss and discrete loss function to quantify the loss during the discretization of continuous minimum hash signature vectors; The discrete loss function is expressed by the following formula: ； in, Represented as the Frobenius norm; The rounding function represents the rounding to the nearest integer.

5. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 4, characterized in that, The discrete loss function is processed using a Straight-Through Estimator during gradient calculation. Non-differentiable problem.

6. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 1, characterized in that, The structured data corresponding to the industrial parts includes the part name, model specifications, manufacturer, material code, purchase price, and installation instructions.

7. The efficient image retrieval method for mineral processing equipment accessories based on deep hash learning according to claim 1, characterized in that, The query in the parts feature database based on the minimum hash code includes: matching all minimum hash codes stored in the parts feature database based on the minimum hash code combined with cosine similarity and hash collision detection mechanism.

Images