Image processing method, device and storage medium

Abstract

The present invention discloses an image processing method, device and storage medium. The method comprises: obtaining an image to be detected, identifying the image to be detected by using a feature extraction model, and obtaining a first feature; matching the first feature with at least one feature in a feature retrieval library, and determining a first distance between the first feature and each of the at least one feature; adjusting the first distance corresponding to each of the features based on a weight; and determining the type of the image to be detected according to the adjusted first distance corresponding to each of the features.

Description

Technical Field

[0001] The present invention relates to the field of network operation and maintenance, and in particular to an image processing method, device and storage medium. Background Art

[0002] With the continuous development of the times, the continuous progress of science, and the accelerating pace of life, image recognition is increasingly used in life, for example, product classification based on image recognition.

[0003] In the related art, a deep neural classification network is trained with a large number of images to obtain a classification model. Due to the large number of training images and the long training process, the classification model types are relatively fixed and cannot be easily changed or replaced. These shortcomings limit the scalability of the settlement method, thereby reducing the commercial value of the settlement method based on image recognition. Summary of the invention

[0004] In view of this, the main purpose of the present invention is to provide an image processing method, device and storage medium.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides an image processing method, the method comprising:

[0007] Acquire an image to be detected, and use a feature extraction model to identify the image to be detected to obtain a first feature;

[0008] Matching the first feature with at least one feature in a feature search library, and determining a first distance between the first feature and each of the at least one feature;

[0009] Adjusting the first distance corresponding to each of the features based on the weight;

[0010] The type of the image to be detected is determined according to the adjusted first distance corresponding to each of the features.

[0011] In the above solution, the method further includes: determining a weight corresponding to each of the features; the determining a weight corresponding to each of the features includes:

[0012] Acquire a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample;

[0013] The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0014] In the above scheme, the number of the types is at least two;

[0015] Determining the similarity between each type of training sample and other types of training samples except itself includes:

[0016] Calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type;

[0017] A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

[0018] In the above solution, the feature retrieval library includes: at least one feature and a type corresponding to each feature;

[0019] The adjusting the first distance corresponding to each of the features based on the weight includes:

[0020] Determine a first distance and a third type corresponding to each of the features;

[0021] Determine a weight corresponding to the third type, and perform weighted calculation on the first distance according to the weight corresponding to the third type to obtain a weighted first distance as the second distance;

[0022] Determine the target feature according to the second distance corresponding to each of the features;

[0023] The type corresponding to the target feature is used as the type of the image to be detected.

[0024] In the above solution, the method further includes: generating the feature extraction model; the generating the feature extraction model includes:

[0025] Get the preset neural network;

[0026] Using the training sample set and the target loss function, the preset neural network is trained to obtain the feature extraction model;

[0027] The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to weights corresponding to training samples of corresponding types.

[0028] An embodiment of the present invention provides an image processing device, the device comprising:

[0029] A first processing module is used to obtain an image to be detected, identify the image to be detected using a feature extraction model, and obtain a first feature;

[0030] A second processing module, configured to match the first feature with at least one feature in a feature search library, and determine a first distance between the first feature and each of the at least one feature;

[0031] The third processing module is used to adjust the first distance corresponding to each of the features based on the weight; and determine the type of the image to be detected according to the adjusted first distance corresponding to each of the features.

[0032] In the above solution, the device further includes: a preprocessing module, used to determine the weight corresponding to each of the features;

[0033] The preprocessing module is specifically used to obtain a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample;

[0034] The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0035] In the above scheme, the number of the types is at least two;

[0036] The preprocessing module is used to extract the features of each training sample in the training sample subset of each type;

[0037] Calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type;

[0038] A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

[0039] In the above solution, the feature retrieval library includes: at least one feature and a type corresponding to each feature;

[0040] The third processing module is used to determine the first distance and the third type corresponding to each of the features;

[0041] Determine a weight corresponding to the third type, and perform weighted calculation on the first distance according to the weight corresponding to the third type to obtain a weighted first distance as the second distance;

[0042] Determine the target feature according to the second distance corresponding to each of the features;

[0043] The type corresponding to the target feature is used as the type of the image to be detected.

[0044] In the above solution, the device further comprises: a model training module, used to generate the feature extraction model;

[0045] The model training module is specifically used to obtain a preset neural network;

[0046] Using the training sample set and the target loss function, the preset neural network is trained to obtain the feature extraction model;

[0047] The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to weights corresponding to training samples of corresponding types.

[0048] An embodiment of the present invention provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above image processing methods when executing the program.

[0049] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above image processing methods are implemented.

[0050] An image processing method, device and storage medium provided by an embodiment of the present invention include: acquiring an image to be detected, using a feature extraction model to identify the image to be detected, and obtaining a first feature; matching the first feature with at least one feature in a feature retrieval library, and determining a first distance between the first feature and each of the at least one feature; adjusting the first distance corresponding to each of the features based on a weight; determining the type of the image to be detected based on the adjusted first distance corresponding to each of the features; in this way, the similarity (i.e., distance) is automatically adjusted through the weight to achieve an accurate matching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a flow chart of an image processing method provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a flow chart of another image processing method provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a dinner plate of different colors and shapes provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of a model training process provided by an embodiment of the present invention;

[0055] Figure 5A schematic diagram of a feature retrieval library establishment process provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of an adaptive feature matching process provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0058] Figure 8 A schematic diagram of the structure of another image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail in conjunction with the embodiments, and the related technology will be described first.

[0060] As mentioned above, image recognition and image classification technologies are widely used in our daily lives, for example, in restaurant checkout and billing.

[0061] At present, many company cafeterias and fast food restaurants adopt the billing method of self-selection and unified checkout in order to speed up the process. The relevant technologies mainly use the following methods to carry out checkout and billing.

[0062] Manual settlement: dishes are identified manually, and then the fees are calculated and settled manually or by a cashier. The settlement speed is slow, and when there are too many checkout personnel, errors are prone to occur, further affecting the settlement rate.

[0063] Barcode-based settlement: Print a barcode corresponding to the dish price on the plate, and scan the barcode to automatically obtain the dish price and settle the bill. The accuracy is greatly improved compared to manual settlement, but the efficiency is still not high because the barcodes of the dishes need to be scanned one by one, and additional hardware such as barcode scanners are required.

[0064] Settlement based on radio frequency identification technology (RFID): an RFID chip is embedded in the plate, the price of the relevant dishes is stored in the chip, and the settlement is completed at the terminal by identifying the chip content. It has high accuracy, fast speed and good settlement efficiency. However, it is necessary to purchase special plates and reading terminals, which is costly, and the plates with embedded chips are easily damaged.

[0065] Settlement based on image recognition: The existing settlement method based on image recognition associates the color, shape, pattern and other features of dishes or plates with the price of dishes, obtains the corresponding dish price through image recognition method at the billing terminal and settles. However, the traditional method uses classification models for recognition, with complex algorithms and single recognition features, which are easily affected by factors such as light, angle and plate occlusion. This results in low recognition rate and easy errors. In addition, the types of plates or dishes are fixed, and the system is not scalable, which affects the commercial value of the system.

[0066] For image recognition-based settlement, the existing image recognition-based settlement method mainly relies on deep learning image classification algorithms, which train deep neural classification networks by collecting a large number of pictures of different plates or dishes to classify different plates or dishes. Due to the large number of training pictures and the long training process, the classification model types are relatively fixed and cannot be easily changed or replaced. These shortcomings limit the scalability of the settlement method, thereby reducing the commercial value of the image recognition-based settlement method.

[0067] Based on this, the method provided by an embodiment of the present invention obtains an image to be detected, uses a feature extraction model to identify the image to be detected, and obtains a first feature; matches the first feature with at least one feature in a feature retrieval library, and determines a first distance between the first feature and each of the at least one feature; adjusts the first distance corresponding to each of the features based on a weight; and determines the type of the image to be detected based on the adjusted first distance corresponding to each of the features.

[0068] The present invention will be further described in detail below in conjunction with the embodiments.

[0069] Figure 1 A flowchart of an image processing method provided by an embodiment of the present invention; Figure 1 As shown, the method is applied to a smart device; the smart device may be a server, a computer, etc. The method includes:

[0070] Step 101: Acquire an image to be detected, use a feature extraction model to identify the image to be detected, and obtain a first feature;

[0071] Step 102: Match the first feature with at least one feature in a feature search library, and determine a first distance between the first feature and each of the at least one feature;

[0072] Step 103: adjusting the first distance corresponding to each of the features based on the weight;

[0073] Step 104: Determine the type of the image to be detected according to the adjusted first distance corresponding to each of the features.

[0074] In some embodiments, before adjusting the first distance corresponding to each of the features based on the weight, the method further includes: determining the weight corresponding to each of the features;

[0075] The determining of the weight corresponding to each of the features includes:

[0076] Acquire a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample;

[0077] The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0078] In some embodiments, the number of types is at least two;

[0079] Determining the similarity between each type of training sample and other types of training samples except itself includes:

[0080] Calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type;

[0081] A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

[0082] In this way, the similarity corresponding to each type can be calculated as the weight of the feature corresponding to each type. For example, when extracting features from the sample images of each type of training sample, when the extracted features belong to the feature retrieval library, the weight of the type of training sample corresponding to the features is the weight corresponding to the features.

[0083] Here, each of the training samples may include: a sample image and a type label corresponding to the sample image.

[0084] Correspondingly, the calculating of the similarity between at least one training sample of the first type and at least one training sample of the second type is to calculate the similarity between sample images included in the training samples of different types.

[0085] The similarity here includes: color similarity, shape similarity, etc.

[0086] For color similarity, color difference can be used to calculate color similarity. For details, refer to the color difference calculation formula of the following formula (1):

[0087]

[0088] Among them, C 1,R , C 1,G , C 1,B Represents the red, green, and blue values ​​of sample 1. 2,R , C 2,G , C 2,B They represent the red, green, and blue values ​​of sample 2. ΔC is the color difference between sample 1 and sample 2.

[0089] For shape similarity, Hu-moments can be used to calculate the similarity of the shapes of two objects.

[0090] In practical applications, the similarity between the image of a certain type of object and the images of other types of objects can be calculated respectively, and the sum of the similarities can be divided by the sum of the similarities between the images of all types of objects to obtain the weight of the above-mentioned certain type of object.

[0091] For example, for a certain object classification, the object has 3 different types, recorded as L1, L2, and L3; type L1 corresponds to 3 object images, recorded as L11, L12, and L13; type L2 corresponds to 2 object images, recorded as L21 and L22; type L3 corresponds to 4 object images, recorded as L31, L32, L33, and L34;

[0092] Assume that the color similarity is calculated, and use the above similarity calculation formula to calculate:

[0093] The similarity between the color of L11 and the colors of L21, L22, L31, L32, L33, and L34; take the average of the similarity results calculated separately and record it as X1;

[0094] The similarity between the color of L12 and the colors of L21, L22, L31, L32, L33, and L34; take the average of the similarity results calculated separately and record it as X2;

[0095] The similarity between the color of L13 and the colors of L21, L22, L31, L32, L33, and L34; take the average of the similarity results calculated separately and record it as X3;

[0096] Then, the weight corresponding to type L1 may be (X1+X2+X3) / 3.

[0097] Correspondingly, for type L2, the above similarity calculation formula is used to calculate:

[0098] The similarity between the color of L21 and the colors of L11, L12, L13, L31, L32, L33, and L34; take the average of the similarity results calculated separately and record it as X4;

[0099] The similarity between the color of L22 and the colors of L11, L12, L13, L31, L32, L33, and L34; take the average of the similarity results calculated separately and record it as X5;

[0100] Then, the weight corresponding to type L2 may be (X4+X5) / 2.

[0101] And so on, calculate the similarity of each type.

[0102] The method for calculating shape similarity is the same as the method for calculating color similarity, and will not be described in detail.

[0103] If the similarity is calculated for both color and shape, the final weight can be the average of the similarities calculated for color and shape. For example, for color, the final similarity calculated is Y1, and for shape, the final similarity calculated is Y2, then the final weight can be (Y1+Y2) / 2.

[0104] After calculating the similarity, the similarity result can be further normalized. Normalization is not limited. For example, for the similarity S1 of type L1, the similarity S2 of type L2, and the similarity S3 of type L3, the normalized result of S1 is S′1=(S1 / (S1+S2+S3)), the normalized result of S2 is S′2=(S2 / (S1+S2+S3)), and the normalized result of S3 is S′3=(S3 / (S1+S2+S3)).

[0105] Furthermore, the weight of type L1 is determined to be S′1, the weight of type L2 is determined to be S′2, and the weight of type L3 is determined to be S′3.

[0106] In some embodiments, determining a first distance between the first feature and each of the at least one feature comprises:

[0107] A preset feature similarity calculation method is used to calculate a first distance between the first feature and each of the at least one feature.

[0108] Here, the features in the feature retrieval library may be features extracted from sample images included in each training sample in the training sample set.

[0109] The preset feature similarity calculation method can be selected according to the requirements. For example, the feature similarity calculation method can adopt the following Cos distance calculation method (refer to the following formula 2) and KL distance calculation method (refer to the following formula 3).

[0110]

[0111]

[0112] Among them, formula 2 represents the cosine distance between the feature values ​​of sample A and sample B. Formula 3 represents the KL distance between sample P and sample Q, where P(x) and Q(x) are the probability distributions of the feature values ​​of sample P and sample Q respectively. The above samples P and Q represent the two samples whose feature similarity needs to be calculated.

[0113] In some embodiments, the feature search library includes: at least one feature and a type corresponding to each of the at least one feature;

[0114] The adjusting the first distance corresponding to each of the features based on the weight includes:

[0115] Determine a first distance and a third type corresponding to each of the features;

[0116] Determine a weight corresponding to the third type, and perform weighted calculation on the first distance according to the weight corresponding to the third type to obtain a weighted first distance as the second distance;

[0117] Determine the target feature according to the second distance corresponding to each of the features;

[0118] The type corresponding to the target feature is used as the type of the image to be detected.

[0119] Here, weighted calculation of the first distance according to the weight corresponding to the third type includes:

[0120] A first distance after multiplication is obtained based on multiplying a preset first coefficient, a weight corresponding to the third type, and the first distance.

[0121] The weights corresponding to the first coefficient and the third type are predetermined.

[0122] The target feature may be selected as the feature with the smallest second distance.

[0123] Here, the features in the feature retrieval library may include features extracted from sample images in each training sample, so the third type corresponding to the feature is also the type corresponding to the sample image from which the feature is extracted. Furthermore, the weight corresponding to the third type is also the weight corresponding to the corresponding sample image.

[0124] The features in the feature retrieval library may also include features extracted from other images using a feature extraction model. Here, the extracted other images correspond to corresponding types, that is, the extracted features may be marked with corresponding types.

[0125] In some embodiments, the method further includes: generating the feature extraction model; the generating the feature extraction model includes:

[0126] Get the preset neural network;

[0127] The preset neural network is trained using the training sample set and the target loss function to obtain the feature extraction model.

[0128] The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to corresponding types of weights.

[0129] Here, the weight can be determined by calculating the similarity of different types of objects. The higher the similarity, the higher the weight (so the weight can be equal to the normalized similarity).

[0130] The preset loss function is weighted by the obtained weight to obtain the target loss function, that is, the target loss function is the result of multiplying the second coefficient, the weight, and the loss function.

[0131] Specifically, the loss function may adopt an edge loss function; different edge loss functions are used in different situations. Here, the edge loss function is a function that takes into account the loss of classification error and the loss of distance between samples.

[0132] For example, the loss function is an additional angle transformation loss function, as shown in the following formula 4:

[0133]

[0134] Among them, N is the number of samples, n is the number of labels, s is the feature normalization parameter, and θ yi is the angle of the yi type (such as the first type, the second type, etc. mentioned above), and m is the minimum angle difference between different types.

[0135] The sum of the second coefficient and the first coefficient is 1. The selection of the second coefficient is set by the developer based on the feature extraction effect. Specifically, if the accuracy of the training result is high, the second coefficient can be smaller (the corresponding first coefficient is larger), and if the accuracy is low, the second coefficient can be larger (the corresponding second coefficient is smaller).

[0136] The preset neural network may adopt MobileNetV2 (a lightweight model).

[0137] The training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample; each of the training samples includes: a sample image and a type corresponding to the sample image (here, the type is used as a label required for training).

[0138] When training the feature extraction model, the sample image and the type label of the sample image are used as input, and the above-mentioned target loss function is used to train the preset neural network (such as MobileNetV2) to obtain the trained neural network as the feature extraction model.

[0139] The feature extraction model obtained through training can perform feature extraction, that is, can recognize images and obtain features of corresponding images as output.

[0140] The image processing method provided by the embodiment of the present invention can be applied to any required scene and is suitable for classification recognition, for example, the recognition of objects in the same category, such as objects of different colors and shapes. It is particularly applicable to the case where the differences between objects in the same category are relatively diverse. For example, dinner plates, backpacks, shoes, etc. of different colors and shapes. Figure 2 The different colors (it should be noted that the colors are not shown in the drawings, but they correspond to different colors in actual application) and different shapes of dinner plates shown; Figure 2 It includes plates of various shapes and colors, as well as the corresponding information of different plates.

[0141] A specific application example is provided below, which is specifically applied to plate recognition. Figure 3 A flowchart of another image processing method provided by an embodiment of the present invention; Figure 3 As shown, the method includes:

[0142] Step 301: Build a feature extraction model.

[0143] Here, we can use MobileNetV2 (a lightweight model) as the basis, compress the MobileNetV2 model, and build a feature extraction model after quantizing the model.

[0144] Step 302: collect pictures of different types of dinner plates as training data, and calculate the training weight of each type of dinner plates according to aspects (color, shape, etc.).

[0145] Here, the weight is determined mainly by calculating the similarity of different types of plates. The higher the similarity, the higher the weight (so the weight can be equal to the similarity).

[0146] The similarity may include: color similarity, shape similarity, etc. For color similarity, a color difference calculation formula may be used, and for shape similarity, a Hu-moments calculation method may be used. The specific calculation formula is in Figure 1 The method is described in detail and will not be repeated here.

[0147] The weight can be obtained by dividing the sum of similarities between one type of plate and other types of plates by the sum of similarities between all plates, and the result is the weight of the above type of plate.

[0148] If only color similarity is used for calculation, then the calculation is performed based on color similarity; if only shape similarity is used, then the calculation is performed based on shape similarity; if both color similarity and shape similarity are used, then the similarity results calculated based on the two similarities can be averaged as the final similarity result. That is, as the weight. Of course, the above division is only one calculation method. In actual applications, other calculation methods can also be used, such as setting different calculation weights for color similarity, shape similarity, etc., and performing weighted calculation based on different calculation weights.

[0149] For example, suppose the training data includes: ten plates; each plate corresponds to a different type; the similarity between each plate and the other nine plates is calculated, the calculated similarity results are added up and divided by nine; or, the similarity between each plate and the other nine plates is calculated, the calculated similarity results are added up and divided by the sum of the similarities between all plates.

[0150] The calculation of specific similarity (including color similarity and shape similarity) and the method of determining training weights based on similarity have been Figure 1 The method is described in detail and will not be repeated here.

[0151] Step 303: determine a preset loss function, perform weighted calculation on the loss function in combination with training weights, perform training based on the weighted loss function and training data, and obtain a feature extraction model.

[0152] Here, the edge loss function is used as the preset loss function. The edge loss function integrates the loss of classification error and distance between samples. Specifically, the following additional angle transformation loss function can be used:

[0153]

[0154] Among them, N is the number of samples, n is the number of labels, s is the feature normalization parameter, and θ yi is the angle of the yi type, and m is the minimum angle difference between different types.

[0155] The weighted calculation of the loss function in combination with the training weights includes:

[0156] The weight obtained by similarity calculation is used to determine a coefficient α (equivalent to Figure 1 The second coefficient in the method shown); where 1>α>0;

[0157] According to the type of plate, the corresponding weight W, α, and loss function are multiplied to obtain the target loss function, and the feature extraction model is trained using the target loss function.

[0158] The feature extraction model can be implemented using any model, that is, according to the training data and the target loss function determined according to the training data, the model used for training is trained, and the trained model obtained can be used as the feature extraction model.

[0159] During training, pictures of different types of plates and the types corresponding to the pictures (the types here are labels) are used as input, and the above-mentioned target loss function is used to train MobileNetV2 to obtain the trained MobileNetV2 as a feature extraction model.

[0160] Here, the model used can be the above-mentioned MobileNetV2, or the MobileNetV2 after model compression.

[0161] For details of steps 301, 302, and 303 related to the model training, please refer to Figure 4 .

[0162] Step 304, using different types of plate images to be used in the restaurant to create a negative library;

[0163] The negative film library here may include the images in the above-mentioned training data, and other dinner plate images may also be collected separately.

[0164] Step 305: Input the images in the film library into the trained feature extraction model, extract the features of different plates, and establish a feature retrieval library.

[0165] The above steps 304 and 305 are the process of establishing the feature retrieval library. Figure 5 That is, for the pictures in the negative library, the feature extraction model is used to extract features, and the extracted features form a feature retrieval library. Among them, each feature corresponds to a corresponding type, and different types correspond to different training weights.

[0166] Step 306: Input the image to be detected into the feature extraction model to extract the corresponding plate features;

[0167] Step 307: Match the plate features to be extracted with the features in the feature retrieval library obtained in step 305, calculate the distance between the features to be retrieved and the features in the feature retrieval library, and then consider the training weight so that the distance can be adaptively adjusted according to the sample type.

[0168] The calculation of the distance between the feature to be retrieved and the feature in the feature retrieval library and then considering the training weights includes adopting one of the following:

[0169] Multiply the calculated distance by the training weight of the sample corresponding to the corresponding matching feature to obtain the final distance result;

[0170] The calculated distance is multiplied by the first coefficient (1-α) and the weight corresponding to the feature to be matched (ie, the training weight of the type to which the feature belongs) to obtain the final distance result.

[0171] The description of calculating the distance between the feature to be retrieved and the feature in the feature retrieval library is as follows;

[0172] Assume that the feature to be retrieved is vector A, and the feature retrieval library includes 10 feature maps, which are respectively recorded as vector B1, vector B2, ... vector B10;

[0173] Calculate the distances between vector A and vector B1, vector B2, ... vector B10 respectively, and obtain distance 1, distance 2, ... distance 10 respectively;

[0174] Taking into account the different similarities of the images, the distance may be incorrect, and a weighted adjustment is performed on the distance, that is, the distance is multiplied by ((1-α)W); wherein W is the weight of the sample type corresponding to the corresponding feature (i.e., the weight calculated in step 301 and step 302); and 1-α is the above-mentioned first coefficient.

[0175] The plate type in the feature retrieval library with the closest weighted distance in the feature space is selected as the plate type of the image to be detected. The above two steps are the adaptive feature matching process. Figure 6 .

[0176] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention; Figure 7 As shown, the device comprises:

[0177] A first processing module is used to obtain an image to be detected, identify the image to be detected using a feature extraction model, and obtain a first feature;

[0178] A second processing module, configured to match the first feature with at least one feature in a feature search library, and determine a first distance between the first feature and each of the at least one feature;

[0179] The third processing module is used to adjust the first distance corresponding to each of the features based on the weight; and determine the type of the image to be detected according to the adjusted first distance corresponding to each of the features.

[0180] Specifically, the device further includes: a preprocessing module, used to determine the weight corresponding to each of the features;

[0181] The preprocessing module is specifically used to obtain a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample;

[0182] The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0183] Specifically, the number of the types is at least two;

[0184] The preprocessing module is used to calculate the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type;

[0185] A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

[0186] Specifically, the feature retrieval library includes: at least one feature and a type corresponding to each feature;

[0187] The third processing module is used to determine the first distance and the third type corresponding to each of the features;

[0188] Determine a weight corresponding to the third type, and perform weighted calculation on the first distance according to the weight corresponding to the third type to obtain a weighted first distance as the second distance;

[0189] Determine the target feature according to the second distance corresponding to each of the features;

[0190] The type corresponding to the target feature is used as the type of the image to be detected.

[0191] Specifically, the device further includes: a model training module, used to generate the feature extraction model;

[0192] The model training module is specifically used to obtain a preset neural network;

[0193] Using the training sample set and the target loss function, the preset neural network is trained to obtain the feature extraction model;

[0194] The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to weights corresponding to training samples of corresponding types.

[0195] It should be noted that: the image processing device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding image processing method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the server is divided into different program modules to complete all or part of the processing described above. In addition, the device provided in the above embodiment and the embodiment of the corresponding method belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0196] Figure 8 A schematic diagram of the structure of another image processing device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the device 80 includes: a processor 801 and a memory 802 for storing a computer program that can be run on the processor; when the processor 801 is used to run the computer program, it executes: obtaining an image to be detected, using a feature extraction model to identify the image to be detected, and obtaining a first feature; matching the first feature with at least one feature in a feature retrieval library, and determining a first distance between the first feature and each of the at least one feature; adjusting the first distance corresponding to each of the features based on a weight; and determining the type of the image to be detected according to the adjusted first distance corresponding to each of the features.

[0197] In one embodiment, the processor 801 is further used to execute the following when running the computer program: obtaining a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample; determining the similarity between each type of training sample and training samples of other types except itself as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0198] In one embodiment, the processor 801 is also used to execute the following when running the computer program: calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type; and determining the weight of the first type based on the similarity between at least one training sample of the first type and at least one training sample of the second type.

[0199] In one embodiment, the processor 801 is also used to execute the following when running the computer program: determining a first distance and a third type corresponding to each of the features; determining a weight corresponding to the third type, performing a weighted calculation on the first distance according to the weight corresponding to the third type, and obtaining a weighted first distance as a second distance; determining a target feature according to the second distance corresponding to each of the features; and using the type corresponding to the target feature as the type of the image to be detected.

[0200] In one embodiment, the processor 801 is also used to execute the following steps when running the computer program: obtaining a preset neural network; using the training sample set and the target loss function to train the preset neural network to obtain the feature extraction model; wherein the target loss function is obtained by weighted calculation of the preset loss function according to the training weights; and the training weights are determined according to the weights corresponding to the corresponding types of training samples.

[0201] In actual application, the device 80 may further include: at least one network interface 803. The various components in the device 80 are coupled together via a bus system 804. It is understood that the bus system 804 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 8 In the figure, various buses are marked as bus system 804. There may be at least one processor 801. The network interface 803 is used for wired or wireless communication between the apparatus 80 and other devices.

[0202] The memory 802 in the embodiment of the present invention is used to store various types of data to support the operation of the device 80 .

[0203] The method disclosed in the above embodiment of the present invention can be applied to the processor 801, or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 801 or the instruction in the form of software. The above processor 801 can be a general processor, a digital signal processor (DSP, DiGital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 801 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 802. The processor 801 reads the information in the memory 802 and completes the steps of the above method in combination with its hardware.

[0204] In an exemplary embodiment, the device 80 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0205] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon; when the computer program is executed by a processor, the following steps are performed: obtaining an image to be detected, identifying the image to be detected using a feature extraction model, and obtaining a first feature; matching the first feature with at least one feature in a feature retrieval library, and determining a first distance between the first feature and each of the at least one feature; adjusting the first distance corresponding to each of the features based on a weight; and determining the type of the image to be detected based on the adjusted first distance corresponding to each of the features.

[0206] In one embodiment, when the computer program is executed by a processor, the following is performed: obtaining a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample; determining the similarity between each type of training sample and training samples of other types except itself as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

[0207] In one embodiment, when the computer program is executed by a processor, the following is performed: calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type; and determining the weight of the first type based on the similarity between at least one training sample of the first type and at least one training sample of the second type.

[0208] In one embodiment, when the computer program is executed by the processor, the following is performed: determining a first distance and a third type corresponding to each of the features; determining a weight corresponding to the third type, performing a weighted calculation on the first distance according to the weight corresponding to the third type, and obtaining a weighted first distance as a second distance; determining a target feature according to the second distance corresponding to each of the features; and using the type corresponding to the target feature as the type of the image to be detected.

[0209] In one embodiment, when the computer program is executed by the processor, the following steps are performed: obtaining a preset neural network; using the training sample set and the target loss function to train the preset neural network to obtain the feature extraction model; wherein the target loss function is obtained by weighted calculation of the preset loss function according to the training weights; and the training weights are determined according to the weights corresponding to the corresponding types of training samples.

[0210] In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0211] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0212] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0213] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0214] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0215] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be detected, and use a feature extraction model to identify the image to be detected to obtain a first feature; Matching the first feature with at least one feature in a feature search library, and determining a first distance between the first feature and each of the at least one feature; wherein the feature search library includes a type corresponding to each of the features; Adjusting the first distance corresponding to each of the features based on the weight; The adjusting the first distance corresponding to each of the features based on the weight includes: Determine a first distance and a third type corresponding to each of the features; Determine a weight corresponding to the third type, and perform weighted calculation on the first distance according to the weight corresponding to the third type to obtain a weighted first distance as the second distance; Determine the target feature according to the second distance corresponding to each of the features; The type corresponding to the target feature is used as the type of the image to be detected.

2. The method according to claim 1, characterized in that The method further includes: determining a weight corresponding to each of the features; the determining a weight corresponding to each of the features includes: Acquire a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample; The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

3. The method according to claim 2, characterized in that The number of said types is at least two; Determining the similarity between each type of training sample and other types of training samples except itself includes: Calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type; A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

4. The method according to any one of claims 2 to 3, characterized in that: The method further includes: generating the feature extraction model; the generating the feature extraction model includes: Get the preset neural network; Using the training sample set and the target loss function, the preset neural network is trained to obtain the feature extraction model; The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to weights corresponding to training samples of corresponding types.

5. An image processing device, characterized in that: The device comprises: A first processing module is used to obtain an image to be detected, identify the image to be detected using a feature extraction model, and obtain a first feature; A second processing module is used to match the first feature with at least one feature in a feature search library, and determine a first distance between the first feature and each of the at least one feature; wherein the feature search library includes a type corresponding to each of the features; The third processing module is used to determine the first distance and the third type corresponding to each of the features; determine the weight corresponding to the third type, perform weighted calculation on the first distance according to the weight corresponding to the third type, and obtain the weighted first distance as the second distance; determine the target feature according to the second distance corresponding to each of the features; and use the type corresponding to the target feature as the type of the image to be detected.

6. The device according to claim 5, characterized in that The device further comprises: a preprocessing module, configured to determine a weight corresponding to each of the features; The preprocessing module is specifically used to obtain a training sample set; the training sample set includes: at least one type of training sample subset; each type of the training sample subset includes: at least one training sample; The similarity between each type of training sample and training samples of other types except itself is determined as the weight of each type; the weight of each type represents the weight of the feature corresponding to each type.

7. The device according to claim 6, characterized in that The number of said types is at least two; The preprocessing module is used to extract the features of each training sample in the training sample subset of each type; Calculating the similarity between at least one training sample of the first type and at least one training sample of the second type; the second type is other types of the at least two types except the first type; A weight of the first type is determined according to a similarity between at least one training sample of the first type and at least one training sample of the second type.

8. The device according to any one of claims 6 to 7, characterized in that The device also includes: a model training module, used to generate the feature extraction model; The model training module is specifically used to obtain a preset neural network; Using the training sample set and the target loss function, the preset neural network is trained to obtain the feature extraction model; The target loss function is obtained by weighted calculation of a preset loss function according to training weights; and the training weights are determined according to weights corresponding to training samples of corresponding types.

9. An image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images