Part type identification method and device, computer equipment and storage medium

Abstract

The invention relates to a part type identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a plurality of part images from a preset comparison image database; the comparison image database comprises images of various parts, and the images of each part comprise images under various shooting conditions; calculating the similarity between each part image and the acquired query image based on a trained target similarity recognition model to obtain a primary similarity set; screening the primary similarity set based on the similarity to obtain a target similarity set; and determining a part type corresponding to each similarity in the target similarity set. By the adoption of the method and device, the accuracy and robustness of part type recognition can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision and image recognition, and in particular to a method, apparatus, computer equipment and storage medium for identifying part types. Background Art

[0002] In high-precision manufacturing fields such as automobile manufacturing, aerospace, and electronic equipment, workers need to Figure 1 Each part shown in the example is identified by type to facilitate classification and management of each part.

[0003] At present, a common method for identifying the type of parts is to manually design the algorithm parameters of the image recognition algorithm according to the specific lighting conditions, shooting posture, shooting angle, shooting viewing distance, shooting occlusion and other shooting conditions when collecting the part image, and input the part image collected under the above-mentioned specific shooting conditions into the image recognition algorithm containing the algorithm parameters for processing, and output the part type.

[0004] However, if the part image is collected under conditions different from the above-mentioned specific shooting conditions, the above-mentioned algorithm parameters are not suitable for the collected part image, which will lead to poor accuracy in part type recognition; and in order to ensure the accuracy of part type recognition, part images can generally only be collected under the above-mentioned specific shooting conditions, which leads to poor robustness of existing part type recognition methods. Summary of the invention

[0005] Based on this, it is necessary to provide a part type identification method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the accuracy and robustness of part type identification.

[0006] In a first aspect, the present application provides a method for identifying a part type, comprising:

[0007] Acquire multiple part images from a preset comparison image database; the comparison image database includes images of multiple parts, and each part image includes images under multiple shooting conditions;

[0008] Based on the trained target similarity recognition model, the similarity between each part image and the collected query image is calculated to obtain a primary similarity set;

[0009] The primary similarity set is screened based on the similarity to obtain the target similarity set;

[0010] Determine the part type corresponding to each similarity in the target similarity set.

[0011] In a second aspect, the present application provides a device for identifying a part type, comprising:

[0012] An acquisition module is used to acquire multiple part images from a preset comparison image database; the comparison image database includes images of multiple parts, and each part image includes images under multiple shooting conditions;

[0013] A calculation module, used for respectively calculating the similarity between each part image and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set;

[0014] A screening module is used to screen the primary similarity set based on the similarity to obtain a target similarity set;

[0015] The determination module is used to determine the part types corresponding to each similarity in the target similarity set.

[0016] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0018] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0019] The above-mentioned part type identification method, device, computer equipment, computer-readable storage medium and computer program product, since the recognition ability of the trained target similarity recognition model has reached the expected level, the accuracy of the calculated similarity can be improved through the target similarity recognition model, and thus the accuracy of the part type determined subsequently can be improved; in addition, the part images processed by the target similarity recognition model are derived from the comparison image database, and the comparison image database includes part images of different types of parts taken under different shooting conditions, which provides a very rich part type comparison graphics for the query image, thereby meeting the need to compare the query image with the part images collected under different shooting conditions, and can improve the robustness of part type recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 An example diagram of a part image provided as background technology for this application;

[0021] Figure 2 An application environment diagram of a method for identifying part types provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a flow chart of a method for identifying a part type provided in an embodiment of the present application;

[0023] Figure 4 An example diagram of a query image provided in an embodiment of the present application;

[0024] Figure 5 An example diagram of a target similarity set provided by an embodiment of the present application, in which each similarity corresponds to a part image in a comparison image database one by one;

[0025] Figure 6 A structural block diagram of a device for identifying part types provided in an embodiment of the present application;

[0026] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0027] Figure 8 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0028] Fig. 9 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The part type identification method provided in the embodiment of the present application can be applied to Figure 2 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0031] like Figure 3 As shown, the embodiment of the present application provides a method for identifying part types, which is applied to Figure 2The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method includes the following steps:

[0032] S310, acquiring multiple part images from a preset comparison image database; the comparison image database includes images of multiple parts, and each part image includes images under multiple shooting conditions.

[0033] Among them, in order to identify the types of parts, this embodiment pre-collects images of multiple parts, and the images of each part include part images under multiple shooting conditions, and the collected part images of each part under multiple shooting conditions are stored in a preset database to obtain a comparison image database; wherein the shooting conditions include one or more of multiple light intensities, multiple light directions, multiple shooting postures, multiple shooting angles, multiple shooting viewing distances and multiple shooting occlusion conditions.

[0034] Exemplarily, in this embodiment, the comparison image database contains a total of 39 types of part images, and the image of each part includes part images under multiple lighting intensities, multiple lighting directions, multiple shooting postures, multiple shooting angles, multiple shooting viewing distances, and multiple shooting occlusion conditions. The comparison image database contains a total of M part images; it should be noted that each part image in the comparison image database is also associated with a corresponding part type.

[0035] Specifically, part images are acquired sequentially from a preset comparison image database, one part image at a time.

[0036] S320 , calculating the similarity between each part image and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set.

[0037] Among them, the query image is an image of a part whose type needs to be identified, and the query image is an image of a part whose type needs to be identified collected under any shooting condition; this embodiment uses a high-resolution industrial camera to collect the part image and the query image respectively; the similarity is used to characterize the degree of similarity between the part image and the query image; the primary similarity set includes the similarity between each part image in the comparison image database and the query image.

[0038] Specifically, the first part image obtained from the comparison image database and the collected query image are input into the target similarity recognition model for processing, and the similarity between the collected query image and the first part image is output; for example, the collected query image is as follows Figure 4As shown; further, a second part image is obtained from the comparison image database, and then the second part image and the query image are input into the target similarity recognition model for processing, and the similarity corresponding to the second part image is output; and so on, the similarity corresponding to each part image in the comparison image database is obtained, and the similarities corresponding to each part image in the comparison image database are collected to obtain a primary similarity set.

[0039] S330 , screening the primary similarity set based on the similarity to obtain a target similarity set.

[0040] The screening is to select several similarities with larger similarity values ​​from the primary similarity set according to the similarity values, and the selected similarities are also the target similarity set.

[0041] S340, determining the part types corresponding to each similarity in the target similarity set.

[0042] Among them, each similarity in the target similarity set corresponds to a part image in the comparison image database, and each part image in the corresponding comparison image database is associated with a corresponding part type, so the part types corresponding to each similarity in the target similarity set can be determined through each similarity in the target similarity set. It should be noted that since the values ​​of each similarity in the target similarity set are large, it means that there is a high degree of similarity between the part images corresponding to each similarity in the comparison image database and the query image, which means that the part types associated with the part images corresponding to each similarity in the target similarity set in the comparison image database are more likely to be the part types of the parts in the query image; determining the part types associated with the part images corresponding to each similarity in the target similarity set in the comparison image database helps subsequent workers to further determine the specific part types of the parts in the query image from the multiple determined part types.

[0043] Specifically, it is determined that each similarity in the target similarity set corresponds to a part image in the comparison image database, such as Figure 5 As shown, the part type associated with each corresponding part image is then further determined.

[0044] It can be seen that in the embodiment of the present application, since the recognition ability of the trained target similarity recognition model has reached the expectation, the accuracy of the calculated similarity can be improved through the target similarity recognition model, and then the accuracy of the part type determined subsequently can be improved; in addition, the part image processed by the target similarity recognition model comes from the comparison image database, and the comparison image database includes part images of different types of parts taken under different shooting conditions, which provides a very rich part type comparison graphics for the query image, thereby meeting the comparison requirements of the query image with the part images collected under different shooting conditions, and can improve the robustness of part type recognition.

[0045] In some embodiments, the trained target similarity recognition model includes a first feature extraction network and a second feature extraction network; based on the trained target similarity recognition model, the similarity between each part image and the collected query image is respectively calculated to obtain a primary similarity set, including:

[0046] S321. Process the part image based on the first feature extraction network to obtain a first feature vector.

[0047] Among them, the target similarity recognition model in this embodiment is specifically a Siamese Neural Network (SNN), and the target similarity recognition model includes an input layer, and the input layer is connected to two feature extraction networks with the same weights and structures, and the two feature extraction networks are respectively recorded as the first feature extraction network and the second feature extraction network, and the two feature extraction networks are connected to a similarity comparison layer; wherein, the input layer is used to input the part image input to the target similarity recognition model into the first feature extraction network, and the first feature extraction network and the second feature extraction network both contain multiple convolutional layers, pooling layers, and nonlinear activation functions, etc. The first feature extraction network is used to gradually extract the feature components of the part image from low-level to high-level, thereby forming a corresponding feature vector, that is, the first feature vector; wherein, the low-level feature components include image edges and image textures, etc., and the high-level feature components include image shapes and image semantics, etc., and the first feature vector is used to comprehensively describe the feature components of the part image.

[0048] S322: Process the collected query image based on the second feature extraction network to obtain a second feature vector.

[0049] Among them, the input layer of the target similarity recognition model is also used to input the query image into the second feature extraction network, and the second feature extraction network is used to gradually extract the feature components of the query image from low-level to high-level, thereby forming a corresponding feature vector, that is, the second feature vector; the second feature vector is used to comprehensively describe the feature components of the query image.

[0050] S323, calculating the similarity between the first feature vector of each part image and the second feature vector of the collected query image, to obtain a primary similarity set.

[0051] Among them, the target similarity recognition model also includes a similarity comparison layer connected to both the first feature extraction network and the second feature extraction network, and the similarity comparison layer is used to calculate the similarity between the first feature vector and the second feature vector; the primary similarity set is a set of similarities between the first feature vectors corresponding to each part image in the comparison image database and the second feature vector corresponding to the query image.

[0052] Specifically, the similarities between the first feature vectors corresponding to each part image in the comparison image database and the second feature vectors corresponding to the query image are calculated respectively, so as to obtain a primary similarity set.

[0053] It can be seen that in this embodiment, the first feature extraction network and the second feature extraction network in the target similarity recognition model can simultaneously perform feature extraction on the part image and the query image, and then compare the first feature vector and the second feature vector extracted through the similarity comparison layer to obtain the similarity between the part image and the query image, that is, the similarity between the part corresponding to the part image and the part corresponding to the query image; by obtaining the similarity between the query image and each part image in the comparison image database, it is convenient to subsequently determine the part image that is relatively similar to the query image from the comparison image database, and then it is convenient to determine the part type corresponding to the query image from the part types corresponding to the relatively similar part images, thereby narrowing the range of part types corresponding to the query image.

[0054] In some embodiments, calculating the similarity between the first feature vector of each part image and the second feature vector of the collected query image to obtain a primary similarity set includes:

[0055] S323A, calculating the dot product of the first eigenvector of each part image and the second eigenvector of the collected query image to obtain a vector dot product.

[0056] The first eigenvector is recorded as F1, the second eigenvector is recorded as F2, and the vector dot product is F1·F2; the first eigenvector F1 includes multiple eigencomponents, and the eigencomponents of the first eigenvector F1 are recorded as F 1i , the second eigenvector F2 contains multiple eigencomponents, and the eigencomponents of the second eigenvector F2 are denoted as F 2i ; The first eigenvector F1 and the second eigenvector F2 have the same number of eigencomponents, and the number of eigencomponents is n.

[0057] Specifically, the dot product of the first eigenvector F1 and the second eigenvector denoted as F2 is calculated to obtain the vector dot product F1·F2.

[0058] S323B, calculating the vector modulus of the first eigenvector of each part image to obtain the first vector modulus, calculating the vector modulus of the second eigenvector of the collected query image to obtain the second vector modulus; calculating the product of each first vector modulus and the second vector modulus to obtain the modulus product.

[0059] Wherein, the vector norm is the length or norm of the corresponding eigenvector, the first vector norm is the vector norm of the first eigenvector F1, and the first vector norm is denoted as ||F1||, The second vector norm is the vector norm of the second eigenvector F2, and the second vector norm is denoted by ||F2||, The modular product is the product of the modularity of the first vector ||F1|| and the modularity of the second vector ||F2|| ||F1||||F2||.

[0060] Specifically, the vector modulus of the first eigenvector F1 is calculated to obtain the first vector modulus ||F1||, the vector modulus of the second eigenvector F2 is calculated to obtain the second vector modulus ||F2||, and then the product of the first vector modulus ||F1|| and the second vector modulus ||F2|| is calculated to obtain the modulus product ||F1||||F2||.

[0061] S323C, determining similarity based on vector dot product and modular product; the primary similarity set includes one-to-one similarities of all part images in the comparison image database.

[0062] Among them, the similarity is the ratio of the vector dot product to the modular product, and the similarity is recorded as s 1i , i represents the i-th part image in the comparison image database, s 1i represents the similarity between the i-th part image in the comparison image database and the query image; and the primary similarity set is denoted as S1, S1 = [s 11 ,s 12 ,……,s 1M ], because there are M part images in the comparison image database, the primary similarity set S1 has M similarities in total.

[0063] Specifically, taking the i-th part image in the comparison image database as an example, the vector dot product F1·F2 and the modular product ||F1||||F2|| between the part image and the query image are obtained. Further, the ratio between the vector dot product F1·F2 and the modular product ||F1||||F2|| is calculated to obtain the similarity s with the i-th part image in the comparison image database. 1i , Need to explain, 1irepresents the cosine value of the angle between the corresponding first eigenvector F1 and the second eigenvector F2; in other implementations, the Euclidean distance or Manhattan distance between the first eigenvector F1 and the second eigenvector F2 can also be collected as the similarity between the first eigenvector F1 and the second eigenvector F2, and the specific similarity is not limited; the similarity s between each part image in the comparison image database and the query image is calculated by the above method 1i , and then collect the similarities s 1i The primary similarity set S1 is obtained.

[0064] It can be seen that in this embodiment, by respectively calculating the vector dot product and the modular product of the first eigenvector and the second eigenvector, and then calculating the ratio of the vector dot product to the modular product, the cosine value of the angle between the first eigenvector and the second eigenvector can be obtained, and the cosine value of the angle is used as the similarity between the corresponding part image and the query image. In this way, the primary similarity set S1 is calculated, which is convenient for preparing for the subsequent determination of part images that are more similar to the query image from the comparison image database.

[0065] In some embodiments, the primary similarity set is screened based on the similarity to obtain a target similarity set, including:

[0066] S331 , sorting the similarities in the primary similarity set based on the numerical order of the similarities to obtain an intermediate similarity set.

[0067] The numerical order in this embodiment is the order from large to small, and in other embodiments it can also be the order from small to large, which is not limited to the specific order; the sorting is to sort the similarities s in the primary similarity set S1. 1i The similarity values ​​are sorted in descending order. The new similarity set obtained after the primary similarity set S1 is sorted is the intermediate similarity set, and the intermediate similarity set is recorded as S2, S2 = [s 21 ,s 22 ,……,s 2M ].

[0068] Specifically, for each similarity s in the primary similarity set S1 1i The similarity values ​​are sorted in descending order to obtain an intermediate similarity set S2.

[0069] S332 , based on a preset number of screenings, a target similarity set is obtained by screening from the intermediate similarity sets in a screening order from large to small.

[0070] The preset number of screening is K. In this embodiment, K is set based on historical experience, and K=5. In other embodiments, the value of K is not specifically limited. The target similarity set is a set of the first K similarities screened from the intermediate similarity set in a screening order from large to small, and the target similarity set is recorded as S3, S3=[s 21 ,……,s 2K ], in this embodiment, S3 = [s 21 ,s 22 ,s 23 ,s 24 ,s 25 ].

[0071] Specifically, a set of top K similarities is selected from the intermediate similarity set, thereby obtaining a target similarity set S3.

[0072] It can be seen that in this embodiment, the determined target similarity set is the set of the top K similarities in the intermediate similarity set S2, and is also the set of the top K highest similarities in the primary similarity set S1. The higher the similarity, the higher the similarity between the part image corresponding to the similarity in the comparison image database and the query image, which means that the possibility that the part type corresponding to the part image is consistent with the part type corresponding to the query image is higher; by screening out the top K similarities, the corresponding K part images in the comparison image database are determined, and then the part types corresponding to the K part images are determined, so as to obtain the range of the part types corresponding to the query image, that is, the part types corresponding to the above K part images; since the value of K is small, the range of the part types corresponding to the query image is small.

[0073] In some embodiments, before respectively calculating the similarity between each part image and the collected query image based on the trained target similarity recognition model to obtain the primary similarity set, the method further includes:

[0074] A110. Based on the primary similarity recognition model, the preset reference sample, positive sample and target negative sample are processed to obtain the positive sample distance and the negative sample distance.

[0075] Among them, the primary similarity recognition model is an untrained similarity recognition model, the benchmark sample is a preset sample that needs to be compared with the positive sample and the target negative sample, and the benchmark sample, the positive sample and the target negative sample are all pre-collected part image samples; wherein the positive sample is a part image sample of the same part type as the benchmark template, and the target negative sample is a part image sample of a different part type from the benchmark sample, and the benchmark sample is denoted as A, the positive sample is denoted as P, and the target negative sample is denoted as N; the primary similarity recognition model is used to process the benchmark sample A and the positive sample P, output the positive sample distance, and the positive sample distance is denoted as d(A, P), and the positive sample distance is used to characterize the similarity between the benchmark sample A and the positive sample P; the primary similarity recognition model is also used to process the benchmark sample A and the target negative sample N, output the negative sample distance, and the negative sample distance is denoted as d(A, N), and the negative sample distance is used to characterize the similarity between the benchmark sample A and the target negative sample N.

[0076] Specifically, the preset benchmark sample A and the positive sample P are input into the preset primary similarity recognition model for processing, and the positive sample distance d(A, P) is output; the preset benchmark sample A and the target negative sample N are also input into the preset primary similarity recognition model for processing, and the negative sample distance d(A, N) is output.

[0077] A120. Calculate the model loss based on the positive sample distance, the negative sample distance, the margin parameter, and the preset comparison threshold.

[0078] Among them, the margin parameter is a preset value, and the margin parameter is recorded as α, the comparison threshold is also a preset value. In this embodiment, the comparison threshold is specifically 0, and in other embodiments, it is not specifically limited; the model loss L(A, P, N) is calculated based on the corresponding model loss function, and the parameters involved in constructing the model loss function include the positive sample distance d(A, P), the negative sample distance d(A, N), the margin parameter α and the preset comparison threshold.

[0079] Specifically, the positive sample distance d(A, P), the negative sample distance d(A, N), the margin parameter α and the preset comparison threshold are substituted into the model loss function for calculation to obtain the corresponding model loss L(A, P, N).

[0080] A130. Iteratively update the primary similarity recognition model based on the model loss and margin parameters until the model loss meets the preset loss value to obtain the target similarity recognition model.

[0081] Among them, when the iterative update, that is, the model loss L(A, P, N) does not reach the preset loss value, the model loss L(A, P, N) is back-propagated in the primary similarity recognition model, thereby updating the model parameters of the primary similarity recognition model to achieve the first round of updating of the primary similarity recognition model. The preset loss value is specifically 0 in this embodiment, and is not specifically limited in other embodiments; then, the primary similarity recognition model after the first round of updating is used as a new primary similarity recognition model, and the above steps A110-A120 are repeatedly executed, thereby achieving iterative training of the primary similarity recognition model until the model loss L(A, P, N) reaches the preset loss value; it should be noted that in the process of repeatedly executing the above steps A110-A120, each time it is executed, it is necessary to consider whether the margin parameter meets the adjustment condition; if the margin parameter meets the adjustment condition, it is necessary to adjust the margin parameter, and obtain a new margin parameter, and apply the new margin parameter to the calculation process of the corresponding model loss L(A, P, N). The new primary similarity recognition model corresponding to the preset loss value is also the target similarity recognition model. If the margin parameter does not meet the adjustment conditions, the margin parameter remains unchanged.

[0082] Specifically, determine whether the model loss L(A, P, N) is a preset loss value. If not, back-propagate the model loss L(A, P, N) in the primary similarity recognition model to update the model parameters of the primary similarity recognition model, thereby obtaining a new primary similarity recognition model; then, repeat the above steps A110-A120 to implement iterative training of the primary similarity recognition model until the model loss L(A, P, N) reaches the preset loss value, and use the new primary similarity recognition model corresponding to the preset loss value as the target similarity recognition model.

[0083] In some embodiments, based on the positive sample distance, the negative sample distance, the margin parameter, and a preset comparison threshold, the model loss is calculated, including:

[0084] A121. Calculate the difference between the positive sample distance and the negative sample distance to obtain the similarity difference.

[0085] Among them, the positive sample distance d(A, P) is used to characterize the distance between the feature vector of the positive sample and the feature vector of the reference sample, and the negative sample distance d(A, N) is used to characterize the distance between the feature vector of the target negative sample and the feature vector of the reference sample; the above distance is specifically one of the Euclidean distance, Manhattan distance and cosine value of the angle between the two feature vectors; the similarity difference is the difference between the positive sample distance d(A, P) and the negative sample distance d(A, N), that is, d(A, P)-d(A, N);

[0086] Specifically, the difference between the positive sample distance d(A, P) and the negative sample distance d(A, N) is calculated to obtain the similarity difference d(A, P)-d(A, N).

[0087] A122. Calculate the sum of the similarity difference and the margin parameter to obtain a comparison parameter.

[0088] Among them, the comparison parameter is the sum of the similarity difference d(A, P)-d(A, N) and the margin parameter α, that is, d(A, P)-d(A, N)+α; the margin parameter α is a constant greater than 0, and in this embodiment, the value range of the margin parameter α is [0.1, 2.0].

[0089] Specifically, the sum of the similarity difference d(A, P)-d(A, N) and the margin parameter α is calculated to obtain the comparison parameter d(A, P)-d(A, N)+α.

[0090] A123. Based on the comparison parameter, the preset comparison function and the preset comparison threshold, the model loss is calculated.

[0091] Among them, the preset comparison function is specifically a maximum value search function max[*] in this embodiment, the comparison threshold is a preset value, in this embodiment, the comparison threshold is specifically 0, and is not specifically limited in other embodiments, and the comparison function max[*] is used to determine the maximum value between the comparison parameter d(A, P)-d(A, N)+α and the comparison threshold; the calculation formula of L(A, P, N) is as follows:

[0092] L(A,P,N)=max[d(A,P)-d(A,N)+α,0];

[0093] Among them, if the comparison parameter d(A, P)-d(A, N)+α is not greater than the comparison threshold 0, the model loss L(A, P, N) is 0, indicating that the primary similarity recognition model does not need to continue to update the model parameters, and the model performance is better at this time; if the comparison parameter d(A, P)-d(A, N)+α is greater than the comparison threshold 0, the model loss L(A, P, N) is d(A, P)-d(A, N)+α, indicating that the target part type recognition model still needs to continue to update the model parameters.

[0094] Specifically, the comparison parameter d(A, P)-d(A, N)+α and the comparison threshold 0 are substituted into the comparison function max[*] for calculation, thereby obtaining the model loss L(A, P, N).

[0095] It can be seen that in this embodiment, when the model loss L(A, P, N) is 0, it means that the smaller the positive sample distance d(A, P), and the larger the negative sample distance d(A, N), that is, the primary similarity recognition model at this time has a strong ability to accurately distinguish positive samples from target negative samples, so the model parameters do not need to be updated further; when the model loss L(A, P, N) is d(A, P)-d(A, N)+α greater than 0, it means that the positive sample distance d(A, P) must be greater than the negative sample distance d(A, N), which also means that the primary similarity recognition model at this time has a weak ability to accurately distinguish positive samples from target negative samples, so the model parameters still need to be updated. By calculating the model loss L(A, P, N), it is easy to determine whether the model parameters need to be updated further.

[0096] In some embodiments, before processing the preset reference sample, the positive sample, and the target negative sample based on the primary similarity recognition model to obtain the positive sample distance and the negative sample distance, the method further includes:

[0097] Calculate the similarity between the preset positive sample and the negative sample based on the primary similarity recognition model to obtain the positive and negative similarity;

[0098] In response to the positive-negative similarity being not less than a preset determination threshold, the negative sample is regarded as a difficult sample, and a new negative sample is obtained based on the difficult sample and the negative sample;

[0099] Based on the primary similarity recognition model, the similarity between the positive sample and the new negative sample is calculated to obtain the new positive and negative similarity;

[0100] In response to the new positive-negative similarity being less than a preset determination threshold, the negative sample is taken as a target negative sample.

[0101] Among them, in order to facilitate improving the ability of the target similarity recognition model to distinguish between the part image and the query image, that is, to improve the ability of the target similarity recognition model to accurately calculate the similarity between the part image and the query image, when training the primary similarity recognition model, the number of negative samples with a high similarity to the positive sample can be increased; to this end, the preset positive sample and negative sample can be first input into the primary similarity recognition model for processing, and the similarity between the positive sample and the negative sample, that is, the positive and negative similarity, is output; the determination threshold is used to compare with the positive and negative similarity to determine whether the positive and negative similarity is high, and the determination threshold is set based on historical experience; the difficult sample is a negative sample corresponding to the positive and negative similarity that is not less than the preset determination threshold, and the difficult sample is also a negative sample with a high degree of similarity to the positive sample; the new negative sample is also a combination of the difficult sample and the negative sample; then, the new negative sample and the positive sample are input into the primary similarity recognition model again for processing, so as to obtain a new positive and negative similarity; and so on, until it is determined that the new positive and negative similarity is less than the preset determination threshold; the target negative sample is also a new negative sample obtained when it is determined that the new positive and negative similarity is less than the preset determination threshold.

[0102] Specifically, the preset positive sample and negative sample are input into the primary similarity recognition model for processing, and the positive and negative similarities are output; then it is determined whether the positive and negative similarities are not less than the preset judgment threshold; if so, the negative sample is used as a difficult sample, and then the difficult sample is combined with the negative sample to obtain a new negative sample; further, the new negative sample and the positive sample are continuously input into the primary similarity recognition model for processing, and the new positive and negative similarities are output; and so on, until it is determined that the new positive and negative similarities are less than the preset judgment threshold; and the new negative sample obtained when it is determined that the new positive and negative similarities are less than the preset judgment threshold is used as the target negative sample.

[0103] It can be seen that in this embodiment, since multiple rounds of adding difficult samples to negative samples will be performed before determining that the new positive and negative similarities are less than the preset judgment threshold, there will be a large number of difficult samples in the target negative samples, and difficult samples help to improve the ability of the trained target similarity recognition model to accurately calculate the similarity between the part image and the query image.

[0104] In some embodiments, the method further comprises:

[0105] In the process of iteratively updating the primary similarity recognition model based on the model loss and the margin parameter, in response to the margin parameter satisfying a preset dynamic update condition, the margin parameter is updated to obtain a new margin parameter; or,

[0106] In response to the margin parameter not satisfying the preset dynamic update condition, the margin parameter is maintained unchanged.

[0107] The dynamic update condition in this embodiment is specifically as follows: taking the initial margin parameter α as the benchmark, every time the primary similarity recognition model is trained for 20 rounds, the value of the margin parameter α is increased once with a fixed increase, thereby obtaining a new margin parameter α. The new margin parameter α is used to be added to the calculation process of the model loss obtained by the next round of training; in this embodiment, the initial margin parameter α is 0.1, and the fixed increase is 0.3.

[0108] Specifically, in the process of iteratively updating the primary similarity recognition model based on the model loss and the margin parameter, it is determined whether the model loss generated by each iterative update reaches the preset loss value. If not, it indicates that the primary similarity recognition model needs to undergo the next round of training. Further, it is determined whether the margin parameter reaches the preset dynamic update condition. If so, the value of the margin parameter α is increased once by a fixed increase to obtain a new margin parameter α. The model loss and the new margin parameter α continue to iteratively update the primary similarity recognition model. If it is determined that the margin parameter does not reach the preset dynamic update condition, the current margin parameter α is maintained unchanged, and the model loss and the current margin parameter α continue to iteratively update the primary similarity recognition model.

[0109] It can be seen that in this embodiment, in the process of iteratively updating the primary similarity recognition model, the value of the margin parameter α will also increase with the increase in the number of training rounds; that is, in the process of initial training of the primary similarity recognition model, the value of the margin parameter α is small, which helps to help the primary similarity recognition model converge quickly; as the training process continues to deepen, the margin parameter α gradually increases, and gradually increasing the margin parameter α helps to improve the ability of the primary similarity recognition model to distinguish between the part image and the query image, that is, to improve the ability of the primary similarity recognition model to calculate the accuracy of the similarity between the part image and the query image. The above steps can realize the dynamic adjustment of the margin parameter α when training the primary similarity recognition model, thereby improving the training effect of training the primary similarity recognition model.

[0110] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0111] Based on the same inventive concept, the embodiment of the present application also provides a part type identification device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the part type identification device provided below can refer to the limitations of the part type identification method above, and will not be repeated here.

[0112] like Figure 6 As shown, the embodiment of the present application provides a part type identification device 600, including:

[0113] The acquisition module 610 is used to acquire multiple part images from a preset comparison image database; the comparison image database includes images of multiple parts, and each part image includes images under multiple shooting conditions;

[0114] A calculation module 620 is used to calculate the similarity between each part image and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set;

[0115] A screening module 630 is used to screen the primary similarity set based on the similarity to obtain a target similarity set;

[0116] The determination module 640 is used to determine the part type corresponding to each similarity in the target similarity set.

[0117] In some embodiments, the trained target similarity recognition model includes a first feature extraction network and a second feature extraction network; in terms of respectively calculating the similarity between each part image and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set, the calculation module 620 is specifically used to:

[0118] Process the part image based on a first feature extraction network to obtain a first feature vector;

[0119] Processing the collected query image based on the second feature extraction network to obtain a second feature vector;

[0120] The similarities between the first feature vectors of each part image and the second feature vectors of the collected query image are calculated to obtain a primary similarity set.

[0121] In some embodiments, in calculating the similarity between the first feature vector of each part image and the second feature vector of the collected query image to obtain a primary similarity set, the calculation module 620 is specifically used to:

[0122] Calculate the dot product of the first eigenvector of each part image and the second eigenvector of the collected query image to obtain a vector dot product;

[0123] Calculate the vector modulus of the first eigenvector of each part image to obtain a first vector modulus, calculate the vector modulus of the second eigenvector of the collected query image to obtain a second vector modulus; calculate the product of each first vector modulus and the second vector modulus to obtain a modulus product;

[0124] The similarity is determined based on the vector dot product and modular product; the primary similarity set includes the one-to-one similarities of all part images in the comparison image database.

[0125] In some embodiments, in terms of filtering the primary similarity set based on the similarity size to obtain the target similarity set, the filtering module 630 is specifically used to:

[0126] The similarities in the primary similarity set are sorted based on the numerical order of the similarities to obtain an intermediate similarity set;

[0127] Based on the preset number of filters, the target similarity set is obtained by filtering from the intermediate similarity set in a descending order.

[0128] In some embodiments, the part type identification device further includes a model training module, which is used to:

[0129] Based on the primary similarity recognition model, the preset benchmark samples, positive samples and target negative samples are processed to obtain the positive sample distance and the negative sample distance;

[0130] The model loss is calculated based on the positive sample distance, negative sample distance, margin parameters and the preset comparison threshold;

[0131] The primary similarity recognition model is iteratively updated based on the model loss and margin parameters until the model loss meets the preset loss value to obtain the target similarity recognition model.

[0132] In some embodiments, in terms of calculating the model loss based on the positive sample distance, the negative sample distance, the margin parameter, and the preset comparison threshold, the model training module is specifically used to:

[0133] Calculate the difference between the positive sample distance and the negative sample distance to get the similarity difference;

[0134] Calculate the sum of the similarity difference and the margin parameter to obtain the comparison parameter;

[0135] Based on the comparison parameters, the preset comparison function and the preset comparison threshold, the model loss is calculated.

[0136] In some embodiments, the part type identification device further includes a sample generation module, and the sample generation module is used to:

[0137] Calculate the similarity between the preset positive sample and the negative sample based on the primary similarity recognition model to obtain the positive and negative similarity;

[0138] In response to the positive-negative similarity being not less than a preset determination threshold, the negative sample is regarded as a difficult sample, and a new negative sample is obtained based on the difficult sample and the negative sample;

[0139] Based on the primary similarity recognition model, the similarity between the positive sample and the new negative sample is calculated to obtain the new positive and negative similarity;

[0140] In response to the new positive-negative similarity being less than a preset determination threshold, the negative sample is taken as a target negative sample.

[0141] In some embodiments, the part type identification device further includes a parameter adjustment module, wherein the parameter adjustment module is used to:

[0142] In the process of iteratively updating the primary similarity recognition model based on the model loss and the margin parameter, in response to the margin parameter satisfying a preset dynamic update condition, the margin parameter is updated to obtain a new margin parameter; or,

[0143] In response to the margin parameter not satisfying the preset dynamic update condition, the margin parameter is maintained unchanged.

[0144] Each module in the above-mentioned part type identification device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0145] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data involved in the part type identification method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned part type identification method are implemented.

[0146] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned part type identification method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0147] Those skilled in the art will understand that Figure 7 or Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0149] In some embodiments, Fig. 9 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0150] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying part types, characterized in that: include: Acquire multiple part images from a preset comparison image database; the comparison image database includes images of multiple parts, and each of the images of the part includes images under multiple shooting conditions; Based on the trained target similarity recognition model, respectively calculate the similarity between each of the part images and the collected query image to obtain a primary similarity set; The primary similarity set is screened based on the similarity to obtain a target similarity set; Determine the part type corresponding to each similarity in the target similarity set.

2. The method according to claim 1, characterized in that The trained target similarity recognition model includes a first feature extraction network and a second feature extraction network; The similarity between each of the part images and the collected query image is calculated based on the trained target similarity recognition model to obtain a primary similarity set, including: Processing the part image based on the first feature extraction network to obtain a first feature vector; Processing the collected query image based on the second feature extraction network to obtain a second feature vector; The similarities between the first feature vectors of each of the part images and the second feature vectors of the collected query image are calculated to obtain a primary similarity set.

3. The method according to claim 2, characterized in that The calculating the similarity between the first feature vector of each of the part images and the second feature vector of the collected query image to obtain a primary similarity set includes: Calculating the dot product of the first feature vector of each of the part images and the second feature vector of the collected query image to obtain a vector dot product; Calculate the vector modulus of the first feature vector of each of the part images to obtain a first vector modulus, calculate the vector modulus of the second feature vector of the acquired query image to obtain a second vector modulus; calculate the product of each of the first vector moduli and the second vector modulus to obtain a modulus product; The similarity is determined based on the vector dot product and the modular product; the primary similarity set includes the similarities of all part images in the comparison image database in a one-to-one correspondence.

4. The method according to claim 1, characterized in that: The step of screening the primary similarity set based on the similarity to obtain a target similarity set includes: Sorting the similarities in the primary similarity set based on the numerical order of the similarities to obtain an intermediate similarity set; Based on a preset screening quantity, a target similarity set is obtained by screening from the intermediate similarity sets in a screening order from large to small.

5. The method according to claim 1, characterized in that Before respectively calculating the similarity between each of the part images and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set, the method further includes: Based on the primary similarity recognition model, the preset benchmark samples, positive samples and target negative samples are processed to obtain the positive sample distance and the negative sample distance; Calculate the model loss based on the positive sample distance, the negative sample distance, the margin parameter and a preset comparison threshold; Iteratively updating the primary similarity recognition model based on the model loss and the margin parameter until the model loss meets a preset loss value, thereby obtaining the target similarity recognition model; The step of calculating the model loss based on the positive sample distance, the negative sample distance, the margin parameter and a preset comparison threshold comprises: Calculate the difference between the positive sample distance and the negative sample distance to obtain a similarity difference; Calculating the sum of the similarity difference and the margin parameter to obtain a comparison parameter; Based on the comparison parameters, the preset comparison function and the preset comparison threshold, the model loss is calculated.

6. The method according to claim 5, characterized in that Before the preset reference sample, positive sample and target negative sample are processed based on the primary similarity recognition model to obtain the positive sample distance and the negative sample distance, the method further includes: Calculating the similarity between the preset positive sample and the negative sample based on the primary similarity recognition model to obtain the positive and negative similarities; In response to the positive-negative similarity being not less than a preset determination threshold, taking the negative sample as a difficult sample, and obtaining a new negative sample based on the difficult sample and the negative sample; Calculating the similarity between the positive sample and the new negative sample based on the primary similarity recognition model to obtain new positive and negative similarities; In response to the new positive-negative similarity being less than a preset determination threshold, the negative sample is used as a target negative sample.

7. The method according to claim 5, characterized in that The method further comprises: In the process of iteratively updating the primary similarity recognition model based on the model loss and the margin parameter, in response to the margin parameter satisfying a preset dynamic update condition, updating the margin parameter to obtain a new margin parameter; or, In response to the margin parameter not satisfying a preset dynamic update condition, the margin parameter is maintained unchanged.

8. A device for identifying part types, characterized in that: include: An acquisition module, used for acquiring a plurality of part images from a preset comparison image database; The comparison image database includes images of multiple parts, and each of the images of the parts includes images under multiple shooting conditions; A calculation module, used for respectively calculating the similarity between each of the part images and the collected query image based on the trained target similarity recognition model to obtain a primary similarity set; A screening module, used for screening the primary similarity set based on the similarity to obtain a target similarity set; The determination module is used to determine the part type corresponding to each similarity in the target similarity set.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images