Complex color identification method, computer program product and storage medium

Abstract

The invention provides a complex color identification method, a computer program product and a storage medium. The recognition method comprises the following steps: extracting a feature vector from a to-be-recognized image, wherein the feature vector is a 1 * n vector formed by n physical features; wherein the physical features comprise color features and / or texture features of the to-be-recognized image; inputting the feature vector into a trained color recognition model, and outputting the category and confidence of the color in the to-be-recognized image by the color recognition model; or performing standardization processing on the feature vector to obtain a standardized feature vector, inputting the standardized feature vector into the trained color recognition model, and outputting the category and confidence of the color in the to-be-recognized image by the color recognition model. The method has good generalization performance, can adapt to brightness change, noise and the like of the environment, and improves the precision and stability of color recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a complex color recognition method, a computer program product and a storage medium. Background Art

[0002] Color recognition is an important research direction in the field of computer vision. In real life, color plays a vital role, such as identifying objects and judging the environment. However, computers face many challenges when processing color information, such as lighting changes, uneven color distribution, color similarity, etc. For some scenarios, such as the color recognition of charging cables, many charging cables are mixed colors, such as two white and red charging cables entangled together, and the entangled charging cables present a variety of shapes and postures, which puts high demands on complex color recognition. In color recognition, we face the following major challenges: 1) Lighting changes: Different lighting conditions (such as direct light, diffuse light, fluorescent light, etc.) will cause colors to look different, and computers need to be able to adapt to these changes in order to accurately identify colors; 2) Uneven color distribution: In natural images, colors are often not evenly distributed, and the color of the surface of an object may change due to texture, shadows, or other factors; 3) Color similarity: There may be subtle differences between colors, but in computer processing, these differences may be ignored, leading to incorrect identification; 4) Color mixing and overflow: In image processing, pixels of different colors may be adjacent, thus affecting each other and causing mixing or overflow of color information; 5) Noise and interference: Noise and other interference factors that may be introduced during image acquisition will also affect the accuracy of color recognition.

[0003] Therefore, it is necessary to improve the deficiencies in the prior art. Summary of the invention

[0004] The present invention mainly provides a complex color recognition method, a computer program product and a storage medium. The recognition method has good generalization performance, can adapt to environmental brightness changes, noise, etc., and improve the accuracy and stability of color recognition.

[0005] According to the first aspect, an embodiment provides a method for identifying complex colors. The method comprises:

[0006] Extracting a feature vector from the image to be identified, wherein the feature vector is a 1×n vector composed of n physical features; wherein the physical features include color features and / or texture features of the image to be identified;

[0007] The feature vector is input into a trained color recognition model, and the color recognition model outputs the category and confidence of the color in the image to be recognized; or, the feature vector is standardized to obtain the standardized feature vector, and the standardized feature vector is input into a trained color recognition model, and the color recognition model outputs the confidence of the color in the image to be recognized, and the category of the color in the image to be recognized is determined according to the confidence.

[0008] In one embodiment, extracting a feature vector from the image to be identified includes:

[0009] Extracting a color feature vector of the image to be identified; wherein the color feature vector is a 1×n1 feature vector composed of statistical information of the color of the red channel, the green channel, the blue channel of the image to be identified and / or the channel obtained by transforming the red channel, the green channel, and the blue channel of the image;

[0010] Extracting a texture feature vector of the image to be identified, wherein the texture feature vector is a 1×n2 feature vector composed of the texture features, wherein the texture feature is a feature of a color moment calculated from the position information and color information of the image to be identified, and n=n1+n2;

[0011] The feature vector is composed of the color feature vector, or the feature vector is composed of the color feature vector and the texture feature vector.

[0012] In one embodiment, extracting the color feature vector of the image to be identified includes:

[0013] Performing HSV transformation on the image to be identified to obtain an HSV image;

[0014] The pixel value ranges in the red channel, green channel, blue channel of the image to be identified and the hue channel of the HSV image are respectively divided into a plurality of sub-pixel value intervals; n1 is obtained based on the value obtained by dividing the size of the value range by the size of the sub-pixel value interval;

[0015] The color feature vector is formed by statistical information of each sub-pixel value interval of the red channel, the green channel, the blue channel and the hue channel of the image to be identified.

[0016] In one embodiment, the color feature vector is formed by statistical information of each sub-pixel value interval of the red channel, the green channel, the blue channel and the hue channel of the image to be identified, including:

[0017] Obtaining a first eigenvector of the red channel, the green channel, the blue channel and / or the hue channel; wherein the first eigenvector is obtained based on the probability of pixel values ​​in the red channel, the green channel, the blue channel and the hue channel being in each sub-pixel value interval; wherein the sub-pixel value interval is obtained by dividing the value range of the pixel values ​​in the red channel, the green channel, the blue channel and the hue channel respectively; wherein the width of the sub-pixel value interval is greater than 1;

[0018] Obtaining a second eigenvector of the red channel, the green channel, the blue channel and / or the hue channel; wherein the second eigenvector is obtained based on the average values ​​of pixel values ​​in the red channel, the green channel, the blue channel and the hue channel that are within each of the sub-pixel value intervals;

[0019] The color feature vector is obtained by concatenating the first feature vectors of each channel, or the color feature vector is obtained by concatenating the second feature vectors of each channel, or the color feature vector is obtained by concatenating the first feature vector of each channel and the second feature vector of each channel.

[0020] In one embodiment, extracting the texture feature vector of the image to be identified includes:

[0021] Obtain multiple features of the moment; wherein the expression of the feature of the moment is:

[0022]

[0023] Among them, x, y and Respectively represent the abscissa, the average value of the abscissa, the ordinate and the average value of the ordinate of the pixel point in the image to be identified, f(x, y) represents the gray value of the pixel point, p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R represent the maximum value of the abscissa and the maximum value of the ordinate respectively;

[0024] The texture feature vector is obtained by concatenating the features of a plurality of the moments.

[0025] In one embodiment, extracting the texture feature vector of the image to be identified includes:

[0026] Obtain multiple features of the moment; wherein the expression of the feature of the moment is:

[0027]

[0028] Among them, the x, y and represents the abscissa, the average value of the abscissa, the ordinate and the average value of the ordinate of the pixel in the image to be identified, respectively; f(x, y) represents the gray value of the pixel in the image to be identified; p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R here represent the maximum value of the abscissa and the maximum value of the ordinate, respectively;

[0029] The features of the moments are normalized using the following expression to obtain multiple normalized features η: p,q :

[0030] r = (p + q) / 2;

[0031] The multiple normalized features η p,q The texture feature vector is obtained by splicing.

[0032] In one embodiment, the step of normalizing the feature vector to obtain the normalized feature vector includes:

[0033] The feature vector is normalized using the following expression to obtain the normalized feature vector:

[0034]

[0035] Among them, the X i represents the ith eigenvector, X′ is the ith eigenvector after standardization, μ represents the mean of the ith eigenvector, σ is the standard deviation of the ith eigenvector, n is the dimension of the eigenvector, X i,j Represents the value of the eigenvector of the jth dimension of the i-th eigenvector.

[0036] In one embodiment, the color recognition model is a multilayer perceptron, which includes an input layer, a hidden layer and an output layer; wherein the number of neurons in the input layer of the multilayer perceptron is the same as the dimension of the feature vector, and the dimension of the feature vector is the sum of the dimensions of the vectors included in the feature vector; the number of neurons in the output layer of the multilayer perceptron is the same as the total number of color categories in the image to be recognized, and the colors represented by each neuron in the output layer are different.

[0037] In one embodiment, the color recognition model is trained through the following process:

[0038] Obtain training feature vectors and corresponding annotation data of training sample images;

[0039] Input the training feature vector into a color recognition model, the color recognition model predicts the color in the training sample image to obtain the category of the predicted color and the category probability corresponding to the predicted color; or, perform standardization on the training feature vector to obtain the standardized training feature vector, input the standardized training feature vector into the color recognition model, the color recognition model predicts the color in the training sample image to obtain the category of the predicted color and the confidence corresponding to the predicted color;

[0040] The color recognition model is trained according to a loss function to obtain the trained color recognition model; the loss function is determined according to the confidence and the label value of the color category in the training sample image; the label value is obtained from the annotated data.

[0041] In one embodiment, the loss function is expressed as:

[0042]

[0043] Here, C represents the number of the training sample images; M represents the total number of color categories in the training sample images; represents the confidence of the i-th predicted color in the c-th training sample image; Represents the label value of the category of the i-th color in the c-th training sample image.

[0044] In one embodiment, the color recognition model is trained according to the loss function to obtain the trained color recognition model, including:

[0045] Step 30: Obtaining initial values ​​of weights and biases in the input layer, the hidden layer, and the output layer;

[0046] Step 31: The color recognition model performs the kth forward propagation; after calculating the kth forward propagation The value of

[0047] Step 32: Using the Calculate the gradient vector of each weight and the bias based on the value of ;

[0048] Step 33: Get adaptive learning rate parameters;

[0049] Step 34: performing a k-th back propagation according to the gradient vector and the adaptive learning rate parameter to iteratively update each of the weights and biases;

[0050] Step 35: Repeat steps 30 to 34 until the number of back propagation exceeds a preset threshold, or the absolute value of the difference between the loss function in two adjacent back propagations is less than a set threshold, then stop the iterative update and use the multilayer perceptron at this time as the trained color recognition model.

[0051] In one embodiment, the color recognition model performs a k-th forward propagation, including:

[0052] Perform forward propagation from the input layer to the hidden layer; wherein the expression of the forward propagation from the input layer to the hidden layer is:

[0053]

[0054] in, is the weight from the nth neuron in the input layer to the ith neuron in the hidden layer, is the bias of the i-th neuron in the hidden layer;

[0055] The y i Input to the nonlinear activation function to obtain Among them, the The expression is:

[0056]

[0057] Perform forward propagation from the hidden layer to the output layer; wherein the expression of the forward propagation from the hidden layer to the output layer is:

[0058]

[0059] Among them, the is the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer, is the bias of the hidden layer;

[0060] The output of the output layer is processed by the Softmax function to obtain the Among them, the The expression is:

[0061] i=1,2,…,M.

[0062] In one embodiment, the use of The numerical calculation of the gradient vector of each of the weights and the bias includes:

[0063] The gradient vector is calculated using expressions of the gradient vectors of the weights and the biases; wherein the expressions of the gradient vectors of the weights and the biases are respectively:

[0064]

[0065] Wherein, L represents the loss function; represents the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer; represents the weight from the nth neuron in the input layer to the ith neuron in the hidden layer; i=1, 2, ... K; K is the number of neurons in the hidden layer; n=1, 2 ... D; D represents the dimension of the feature vector; m=1, 2, ... M; M represents the number of neurons in the output layer; represents the rate of change from the weight of the mth neuron in the output layer to the ith neuron in the hidden layer; represents the rate of change of the bias of the mth neuron in the output layer; represents the output value of the i-th neuron in the hidden layer; represents the rate of change from the weight of the i-th neuron in the hidden layer to the n-th neuron in the input layer; represents the rate of change of the bias of the i-th neuron in the hidden layer; the Y k Represents the k-th forward propagation after or

[0066] In one embodiment, obtaining the adaptive learning rate parameter includes:

[0067] The learning rate after the kth forward propagation is calculated according to the following expression

[0068]

[0069] Among them, the g k Represents the k-th forward propagation after or or or The D is a preset coefficient;

[0070] The intermediate variables of each weight are calculated according to the following expression:

[0071]

[0072] Among them, the Y k represents the value of the weight after the k-th forward propagation; It means that after the kth forward propagation or The intermediate variable of

[0073] First use the For the above gradient vector expression, Y k Update, and then use the expression of the gradient vector of each weight and the bias to calculate Among them, the Represents the k-th forward propagation after or or or The intermediate variable of

[0074] The new learning rate is calculated according to the following expression:

[0075]

[0076] Wherein, the adaptive learning rate parameters include the and

[0077] In one embodiment, the iterative updating of each of the weights and biases by performing a k-th back propagation according to the gradient vector and the adaptive learning rate parameter includes:

[0078] Calculated according to the following expression:

[0079]

[0080] Among them, the Y k+1 Represents the value of each weight after the k-th forward propagation;

[0081] First use the Y k+1 For the above gradient vector expression, Y k , and then use the expressions of the gradient vectors of the weights and the biases to calculate g k+1 ; wherein said g k+1 Represents the k+1th forward propagation after or or or

[0082] Calculated according to the following expression:

[0083]

[0084] The β k represents the step size of updating the weight after the k-th forward propagation;

[0085] Calculate g according to the following expression k+2 :

[0086] g k+2 =(-g k+1 )+(-g k )×β k ;

[0087] Among them, the g k+2 Represents the k+1th forward propagation after or or or

[0088] According to a third aspect, an embodiment provides a computer program product, comprising a computer program and / or instructions, wherein when the computer program and / or instructions are executed by a processor, the identification method as described in any one of the embodiments herein is implemented.

[0089] According to a fourth aspect, an embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium includes a program, wherein the program can be executed by a processor to implement a method as described in any embodiment of the present invention.

[0090] The beneficial effects of this application are:

[0091] The recognition method includes: extracting a feature vector from an image to be recognized, the feature vector being a 1×n vector composed of n physical features; wherein the physical features include color features and / or texture features of the image to be recognized; inputting the feature vector into a trained color recognition model, and the color recognition model outputs the category and confidence of the color in the image to be recognized; or, performing standardization processing on the feature vector to obtain a standardized feature vector, inputting the standardized feature vector into a trained color recognition model, and the color recognition model outputs the category and confidence of the color in the image to be recognized; since the feature vector extracted by the recognition method is a high-dimensional feature vector, the feature vector can adapt to the influence of brightness changes and noise; the color recognition model is used to learn the feature vector, and a nonlinear structure is constructed, so that it can adapt to changes in the external environment. Therefore, the method has good generalization performance, can adapt to environmental brightness changes, noise, etc., and improve the accuracy and stability of color recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 A schematic diagram of a flow chart of a complex color recognition method according to an embodiment;

[0093] Figure 2A schematic diagram of a process of extracting a feature vector from an image to be identified according to an embodiment;

[0094] Figure 3 A schematic diagram of a process of extracting a color feature vector of an image to be identified according to an embodiment;

[0095] Figure 4 A schematic diagram of a process of forming a color feature vector according to an embodiment;

[0096] Figure 5 A schematic diagram of a process of extracting a texture feature vector of an image to be identified according to an embodiment;

[0097] Figure 6 A schematic diagram of a process of extracting a texture feature vector of an image to be identified according to another embodiment;

[0098] Figure 7 A schematic diagram of a training process of a color recognition model according to an embodiment;

[0099] Figure 8 A schematic diagram of a process of training a color recognition model according to a loss function to obtain a trained color recognition model according to an embodiment;

[0100] Fig. 9 A schematic diagram of a process for obtaining an adaptive learning rate parameter according to an embodiment;

[0101] Fig.10 A schematic diagram of a flow chart of iteratively updating each weight and bias according to an embodiment;

[0102] Fig.11 A schematic diagram of forward propagation of a color recognition model according to an embodiment. DETAILED DESCRIPTION

[0103] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0104] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0105] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0106] The existing complex color recognition method is to give a pixel value range for the color to be recognized. When the pixel value of a point in the image is within the given value range, it is considered to belong to the recognized color, and finally the grayscale value of the point is set to 255, otherwise it is set to 0. Finally, a binary image is obtained. Traditional complex color recognition methods are mainly based on distance threshold methods for judgment, such as Euclidean distance, block distance or Mahalanobis distance. The distance-based judgment method mainly calculates the distance between the color to be recognized and the corresponding center by learning the mean or standard deviation of the standard sample, and finally selects the category with the closest distance as the corresponding color type result. Although this method is intuitive and easy to understand, it is insufficient in expressing color differences and perception. Methods such as color moments and color histograms can capture the statistical characteristics of colors, but may not be able to distinguish images with similar color distributions but different textures.

[0107] The technical concept of the present application is to use machine learning methods to complete the recognition of complex colors. For example, a multi-layer perceptron is used as a color recognition model to recognize features extracted from the image to be recognized. Among them, the key step is the selection of color features, which is then adapted to the recognition of complex colors.

[0108] The technical solution of the present application will be described in detail below in conjunction with embodiments.

[0109] Please refer to Figure 1 , this application proposes a complex color recognition method. The recognition method includes:

[0110] Step S100: extracting a feature vector from the image to be identified;

[0111] Step S200: Input the feature vector into the color recognition model and output the color category and confidence.

[0112] Specifically, in step S100, the feature vector is a 1×n vector composed of n physical features; wherein the physical features include color features and / or texture features of the image to be identified.

[0113] Specifically, step S200 may include: inputting the feature vector into a trained color recognition model, the color recognition model outputting the confidence of the color in the image to be recognized, and judging the category of the color in the image to be recognized based on the confidence; or, normalizing the feature vector to obtain a standardized feature vector, inputting the standardized feature vector into the trained color recognition model, the color recognition model outputting the confidence of the color in the image to be recognized, and judging the category of the color in the image to be recognized based on the confidence.

[0114] In step S100, those skilled in the art may determine the type of the image to be identified by themselves, and the type of the image to be identified is not specifically limited herein.

[0115] For complex color feature recognition, selecting an appropriate feature vector is key. Color features or color feature vectors need to fully represent the essential attributes of the color in the image to be recognized, while helping to reduce the impact of irrelevant information to facilitate subsequent image processing and analysis. Therefore, in some embodiments, the present application comprehensively considers shape and texture information such as color space, color histogram, color moment, etc. to complete the selection of color features.

[0116] For some examples, please refer to Figure 2 In step S100, extracting a feature vector from the image to be identified includes:

[0117] Step S110: extracting the color feature vector of the image to be identified;

[0118] Step S120: extracting texture feature vectors of the image to be identified;

[0119] Step S130: composing a feature vector from the color feature vector, or composing a feature vector from the color feature vector and the texture feature vector.

[0120] In step S110, the color feature vector is a 1×n1 feature vector composed of statistical information of the red channel, green channel, blue channel of the image to be identified and / or the color obtained by transforming the red channel, green channel, blue channel of the image. In other words, the color feature includes the statistical information of the above color.

[0121] It should be noted that, in some embodiments, if the feature vector is composed only of a color feature vector, then in step S200, the feature vector may not meet the requirement for accurate identification of mixed colors (such as a color mixed with multiple colors such as red and white, red and blue), that is, the color recognition model cannot use the feature vector to accurately identify the mixed color. In other embodiments, if the feature vector is composed of a color feature vector and a texture feature vector, then in step S200, the feature vector can meet the requirement for accurate identification of mixed colors (such as a color mixed with multiple colors such as red and white, red and blue), that is, the color recognition model can use the feature vector to accurately identify the mixed color.

[0122] For some examples, please refer to Figure 3 In step S110, the color feature vector of the image to be identified is extracted, including:

[0123] Step S111: performing HSV transformation on the image to be identified to obtain an HSV image;

[0124] Step S112: dividing the value ranges of pixel values ​​in the red channel, green channel, blue channel of the image to be identified and the hue channel of the HSV image into a plurality of sub-pixel value intervals respectively;

[0125] Step S113: A color feature vector is constructed from statistical information of each sub-pixel value interval of the red channel, green channel, blue channel and hue channel of the image to be identified.

[0126] It should be noted that the HSV transformation in "performing HSV transformation on the image to be identified to obtain an HSV image" in step S111 is a process of converting the image to be identified from the RGB (colors of the three channels of red, green and blue) color space to the HSV color space, where HSV stands for hue, saturation and value. This conversion is usually used in image processing because it provides a more intuitive way to process colors. Since the specific process of "performing HSV transformation on the image to be identified to obtain an HSV image" belongs to the prior art in this field, its specific process will not be repeated here.

[0127] In some embodiments, in step S112, those skilled in the art can determine the size of the above-mentioned sub-pixel value interval according to actual needs. Among them, the above-mentioned value range can be evenly divided into multiple sub-pixel value intervals. For example, the size of the sub-pixel value interval can be equal to 1; the size of the sub-pixel value interval can also be greater than 1. n1 is obtained based on the value obtained by dividing the size of the value range by the size of the sub-pixel value interval. For example, when the above-mentioned "pixel value range" is 0-255 and the size of the sub-pixel value interval is equal to 1, then n1 can be an integer multiple of 256; and if the size of the sub-pixel value interval is equal to 8, then n1 can be an integer multiple of 32; and so on. In addition, the above-mentioned value range can also be unevenly divided into multiple sub-pixel value intervals according to actual needs.

[0128] For some examples, please refer to Figure 4 In step S113, the color feature vector is formed by the statistical information of each sub-pixel value interval of the red channel, green channel, blue channel and hue channel of the image to be identified, including:

[0129] Step S113a: Obtaining a first eigenvector of a red channel, a green channel, a blue channel and / or a hue channel;

[0130] Step S113b: Obtain a second eigenvector of a red channel, a green channel, a blue channel and / or a hue channel;

[0131] Step S113c: concatenate the first eigenvectors of each channel to obtain a color eigenvector, or concatenate the second eigenvectors of each channel to obtain a color eigenvector, or concatenate the first eigenvector of each channel and the second eigenvector of each channel to obtain a color eigenvector.

[0132] In some embodiments, in step S113a, the first feature vector is obtained based on the probability of the pixel value (the pixel value is the gray value at this time) in each sub-pixel value interval in the red channel, the green channel, the blue channel and the hue channel; wherein the sub-pixel value interval is obtained by dividing the value range of the pixel value in the red channel, the green channel, the blue channel and the hue channel respectively. The present application hopes that the color feature vector can adapt to certain rotation and translation invariance, so the above channels can be processed and analyzed in the form of a histogram. In order to adapt to certain environmental changes and illumination changes, the present application does not directly obtain the first feature vector based on the histogram of the above channels, but sets the width of the above sub-pixel value interval to be greater than 1, and uses the barrel statistical information of the above histogram to obtain the first feature vector. The calculation process of the first feature vector is roughly as follows: first obtain the relative histogram of the above channels, obtain the frequency of each gray value appearing in each sub-pixel value interval, divide it by the total number of pixels in the image to be identified, and then obtain the first feature vector. For example, the probabilities of pixel values ​​in the red channel, green channel, and blue channel that are within each sub-pixel value interval can be expressed as:

[0133] H R (i) = N R (i) / N,H G (i) = N G (i) / N,H B (i) = N B (i) / N;

[0134] Where i = 1, ..., 255, N R (i) is the number of pixels with gray value i in the red channel above, N G (i) is the number of pixels with gray value i in the green channel above, N B(i) is the number of pixels of gray value i in the above blue channel, and N is the total number of pixels in the image to be identified. For example, when the size of the sub-pixel value interval is set to 8, each of the above channels can obtain a first feature vector composed of 256 / 8=32 probabilities of pixel values ​​in each sub-pixel value interval, and the dimension of the first feature vector is 32. That is, the first element in the first feature vector represents the probability of a pixel value in the first sub-pixel value interval (such as the range of 0-7) in the corresponding channel, and the second element in the first feature vector represents the probability of a pixel value in the second sub-pixel value interval (such as the range of 8-15) in the corresponding channel, and so on. The probability of a pixel value in each sub-pixel value interval (such as the range of 0-7) is represented by the cumulative sum of the probabilities / frequencies of each gray value in the relative histogram appearing in the sub-pixel value interval. For example, the probability of a pixel value in the sub-pixel value interval of 0-7 is equal to the percentage of the number of gray values ​​in the sub-pixel value interval of 0-7 to the total number of pixels in the entire image. At this time, n1 is an integer multiple of 32, such as 32, 64, 96, 128, etc. That is, the first eigenvectors of the red channel, green channel, blue channel and hue channel respectively count the probabilities of pixel values ​​in the red channel, green channel, blue channel and hue channel being within each sub-pixel value interval.

[0135] It should be noted that in step S113a, the reason for "setting the width of the above sub-pixel value interval to be greater than 1" is: usually the histogram counts the number of occurrences or the probability of each gray value, and it can be understood that the width of the sub-pixel value interval of the histogram is equal to 1 at this time; and the barrel statistical information in this application refers to the number of occurrences or the probability of each gray value in the corresponding sub-pixel value interval; that is, when the external environment undergoes certain changes, although a certain gray value in the image to be identified may have changed, the gray value after the change is likely to still be within the original sub-pixel value interval, therefore, the recognition method of this application can adapt to certain environmental changes and lighting changes. Since the specific process of obtaining the above "probability of pixel values ​​within each sub-pixel value interval" is common knowledge in the art, the specific process of obtaining the first eigenvector will not be repeated here.

[0136] In some embodiments, in step S113b, the second feature vector is obtained based on the average values ​​of pixel values ​​(the pixel values ​​are gray values ​​in this case) in each sub-pixel value interval in the red channel, the green channel, the blue channel and the hue channel. The width of the sub-pixel value interval may be greater than 1.

[0137] It should be noted that in step S113b, the role of the second eigenvector is to increase the expression of statistical information to distinguish / accurately identify similar colors and avoid large differences in data. This is because when the above-mentioned eigenvector includes the second eigenvector, if there are multiple similar colors (such as red, dark red and bright red, etc.) in the image to be identified, although the color distribution of the above-mentioned similar colors in the red channel, green channel and blue channel (such as the distribution of the above-mentioned average value) is relatively close, the color difference / value difference in the hue channel is large, so the above-mentioned eigenvector including the second eigenvector can be used to distinguish or accurately identify colors of the same color system in the image to be identified.

[0138] It should be noted that the specific process of obtaining the second eigenvector in step S113b is roughly the same as the specific process of obtaining the first eigenvector in step S113a, the only difference being that the second eigenvector is obtained based on the average values ​​of the pixel values ​​in each sub-pixel value interval in the red channel, the green channel, the blue channel and the hue channel, respectively, while the first eigenvector is obtained based on the probability of the pixel values ​​in each sub-pixel value interval in the red channel, the green channel, the blue channel and the hue channel, respectively. Since the specific process of obtaining the above-mentioned "average value of the pixel values ​​in each sub-pixel value interval" is common knowledge in the art, the specific process of obtaining the second eigenvector will not be described in detail here.

[0139] In step S113c, those skilled in the art can determine whether to splice the first feature vectors of each channel to obtain the color feature vector, or splice the second feature vectors of each channel to obtain the color feature vector, or splice the first feature vector of each channel and the second feature vector of each channel to obtain the color feature vector according to actual needs. For example, if the size of the sub-pixel value interval is equal to 8, and the color feature vector is obtained by splicing the first feature vectors of the red channel, the green channel, the blue channel and the hue channel, the dimension of the first feature vector is 32, and the dimension of the color feature vector is 32×4=128. For another example, if the size of the sub-pixel value interval is equal to 8, and the color feature vector is obtained by splicing the first feature vector and the second feature vector of the red channel, the green channel, the blue channel and the hue channel, the dimensions of the first feature vector and the second feature vector are both 32, and the dimension of the color feature vector is 32×8=256. Other situations can be deduced by analogy.

[0140] In step S120, the texture feature vector is a 1×n2 feature vector composed of texture features, wherein the texture feature is a feature of a color moment calculated from the position information and color information of the image to be identified, and n=n1+n2.

[0141] In some embodiments, in order to improve the comprehensive expression ability of the feature vector, the feature vector needs to include not only the color feature vector but also the texture feature vector. Figure 5 In step S120, the texture feature vector of the image to be identified is extracted, including:

[0142] Step S121a: obtaining features of multiple moments;

[0143] Step S122a: concatenate the features of multiple moments to obtain a texture feature vector.

[0144] In some embodiments, in step S121a, the expression of the characteristic of the moment is:

[0145]

[0146] Among them, x, y and Respectively represent the abscissa, average value of abscissa, ordinate and average value of ordinate of the pixel in the image to be identified, f(x,y) represents the gray value of the pixel, p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R here represent the maximum value of the abscissa and the maximum value of the ordinate respectively. p.q It can be used as a moment expression of the texture information in the image to be identified. When p = 0 and q = 0, the above m p.q represents the average gray value of each pixel in the image to be identified; when p = 1 and q = 0, the above m p.q represents the first-order central moment of x; and so on for other cases.

[0147] In some embodiments, in step S122a, since p can be 0, 1, or 2, and q can be 0, 1, or 2, those skilled in the art can determine the number of features used for the splicing, such as 1, 2, or 8, according to actual needs. For example, 8 features of the above moments can be selected for splicing to obtain a texture feature vector. The "splicing" operation here is a conventional operation in the art, so it will not be described in detail.

[0148] For some examples, please refer to Figure 6 In step S120, the texture feature vector of the image to be identified is extracted, including:

[0149] Step S121b: Obtain multiple moment features; wherein the expression of the moment feature is:

[0150]

[0151] Among them, x, y and Respectively represent the abscissa, the average value of the abscissa, the ordinate and the average value of the ordinate of the pixel in the image to be identified, f(x,y) represents the gray value of the pixel in the image to be identified; p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R here represent the maximum value of the abscissa and the maximum value of the ordinate respectively;

[0152] Step S122b: Normalize the features of the moments using the following expression to obtain multiple normalized features η: p,q :

[0153] r = (p + q) / 2;

[0154] Step S123b: Multiple normalized features η p,q The texture feature vector is obtained by splicing.

[0155] It should be noted that the contents of step S121b are substantially the same as those of step S121a, and the contents of step S123b are substantially the same as those of step S122a, so they will not be described in detail here.

[0156] It should be noted that the purpose of normalizing the moment features using the zero-order central moment in step S122b is to offset the influence of the central moment caused by scale changes (such as an increase or decrease in the number of pixels in the image). That is to say, when using the normalized features for subsequent color recognition, it can avoid identifying one color as another color when the number of pixels in the image increases or decreases.

[0157] It should be noted that the texture feature vector describes the relationship between the position of the pixel in the image and the grayscale of the pixel. The first feature vector and the second feature vector only consider the statistics of the grayscale, ignoring the relationship between the position and the grayscale. The calculation of the texture feature vector not only considers the spatial position relationship, but also the grayscale information, and performs a correlation analysis on the two.

[0158] Since there are some common problems with raw data, the raw data must be standardized to improve the quality of subsequent data analysis. Therefore, in some embodiments, the feature vector is standardized to obtain a standardized feature vector, including:

[0159] Step S210: Standardize the feature vector using the following expression to obtain a standardized feature vector:

[0160]

[0161] Among them, X irepresents the ith eigenvector, X′ is the ith eigenvector after standardization, μ represents the mean of the ith eigenvector, σ is the standard deviation of the ith eigenvector, n is the dimension of the eigenvector, X i,j Represents the value of the eigenvector of the jth dimension of the i-th eigenvector.

[0162] In some embodiments, the color recognition model is a multi-layer perceptron, which includes an input layer, a hidden layer, and an output layer.

[0163] It should be noted that the multilayer perceptron (MLP) is a basic and widely used artificial neural network model, whose structure consists of multiple levels, and the number of its hidden layers can be one or more. The multilayer perceptron has shown excellent performance in classification, regression, pattern recognition and other fields with its powerful nonlinear mapping ability and flexible structural design. Among them, those skilled in the art can determine the number of hidden layers in the color recognition model by themselves.

[0164] In some embodiments, the number of neurons in the input layer is the same as the dimension of the feature vector, and the dimension of the feature vector is the sum of the dimensions of the vectors included in the feature vector. For example, in the case where the feature vector is composed of a color feature vector and a texture feature vector, if the size of the sub-pixel value interval is equal to 8, and the color feature vector is obtained by splicing the first feature vector and the second feature vector of the red channel, the green channel, the blue channel and the hue channel, then the dimensions of the first feature vector and the second feature vector are both 32, and the dimension of the color feature vector is: 32×(4+4)=256, where the first 4 refers to the number of the first feature vectors, and the second 4 refers to the number of the second feature vectors; and if the texture feature vector is obtained by splicing the features of the 8 moments, then the dimension of the feature vector is: 32×4+32×4+8=264, and the number of neurons in the input layer is also 264.

[0165] In some embodiments, the number of neurons in the output layer of the multilayer perceptron is the same as the total number of color categories in the image to be identified, and the colors represented by each neuron in the output layer are different.

[0166] For some examples, please refer to Figure 7 , the above color recognition model is trained through the following process:

[0167] Step 100: Obtain training feature vectors and corresponding annotation data of training sample images;

[0168] Step 200: input the training feature vector into the color recognition model, and the color recognition model predicts the color in the training sample image to obtain the predicted color category and the category probability corresponding to the predicted color;

[0169] Alternatively, the training feature vector is normalized to obtain a normalized training feature vector, the normalized training feature vector is input into a color recognition model, and the color recognition model predicts the color in the training sample image to obtain a predicted color category and a confidence level corresponding to the predicted color;

[0170] Step 300: Train the color recognition model according to the loss function to obtain a trained color recognition model; the loss function is determined according to the confidence level and the label value of the color category in the training sample image.

[0171] In step 100, those skilled in the art may determine the type of training sample images by themselves, and the type of images to be identified is not specifically limited herein.

[0172] It should be noted that the specific process of obtaining the "training feature vector" in step 100 can be substantially the same as the specific process of obtaining the "feature vector" in step S100, so it will not be repeated here. The specific process of obtaining the "labeled data" belongs to the common knowledge in the art.

[0173] In some embodiments, in step 200, preferably, the training feature vector is standardized to obtain a standardized training feature vector. Thereafter, the standardized training feature vector is input into the color recognition model.

[0174] In some embodiments, in step 300, cross entropy may be used as the loss function. For example, the loss function may be expressed as:

[0175]

[0176] Where C represents the number of training sample images; M represents the total number of color categories in the training sample images; Represents the confidence of the i-th predicted color in the c-th training sample image; Indicates the label value of the category of the i-th color in the c-th training sample image. The label value is obtained from the above-mentioned labeled data. Among them, the label value can be composed of 0 and 1, and the cumulative sum of each label can be 1.

[0177] For some examples, please refer to Figure 8 Step 300: training the color recognition model according to the loss function to obtain a trained color recognition model, including:

[0178] Step 30: Obtain the initial values ​​of the weights and biases in the above input layer, hidden layer and output layer;

[0179] Step 31: The color recognition model performs the kth forward propagation; calculate s after the kth forward propagation i c -l i c The value of

[0180] Step 32: Utilize s i c -l i c The numerical value of Calculate the gradient vector of each weight and bias;

[0181] Step 33: Get adaptive learning rate parameters;

[0182] Step 34: Perform the kth back propagation according to the gradient vector and the adaptive learning rate parameter to iteratively update each weight and bias;

[0183] Step 35: Repeat steps 30 to 34 until the number of back propagation exceeds the preset number threshold, or the absolute value of the difference between the loss function in two adjacent back propagations is less than the set threshold, then stop the iterative update and use the multilayer perceptron at this time as the trained color recognition model.

[0184] It should be noted that in step 30, the process of obtaining the above initial value belongs to common knowledge in the art, so it will not be described in detail here.

[0185] For some examples, please refer to Fig.11 , assuming that the input layer of this color recognition model has 3 neurons, x1, x2 and x3 represent the feature vectors of the 3 neurons in the input layer, y1 to y K Respectively represent the outputs of the 1st to Kth neurons in the hidden layer, z1 to z M Respectively represent the outputs of the 1st to the Mth neurons in the output layer, s1 to s M Respectively represent the confidence of the predicted 1st to Mth colors. Step 31: The color recognition model performs the kth forward propagation, including:

[0186] Step 31a: Perform forward propagation from the input layer to the hidden layer; wherein the expression for forward propagation from the input layer to the hidden layer is:

[0187]

[0188] Among them, K represents the number of neurons in the above hidden layer, N represents the dimension of the above training feature vector, that is, the number of neurons in the above input layer, is the weight from the nth neuron in the input layer to the ith neuron in the hidden layer, is the bias of the i-th neuron in the hidden layer; x n Represents the feature vector of the nth neuron in the input layer (such as the normalized feature vector);

[0189] Step 31b: Place y i Input to the nonlinear activation function to obtain

[0190] Step 31c: Perform forward propagation from the hidden layer to the output layer; wherein the expression for forward propagation from the hidden layer to the output layer is:

[0191]

[0192] Where M represents the number of neurons in the output layer, Z m represents the output of the mth neuron in the output layer, is the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer, is the bias of the hidden layer;

[0193] Step 31d: The output of the output layer is processed by the Softmax activation function to obtain

[0194] In the above step 31b, those skilled in the art can determine the type of the nonlinear activation function according to actual needs. For example, The expression can be:

[0195]

[0196] Among them, tanh in the above formula is a common nonlinear activation function.

[0197] In the above step 31d, the Softmax function is one of the main functions used in the classification model, which is often used for multi-classification in its last layer. The expression can be:

[0198] i=1,2,…,M;

[0199] Among them, Z j represents the output of the jth neuron in the output layer, Represents the confidence of the i-th predicted color in the c-th training sample image.

[0200] In the above step 32, using s i c -li c The numerical calculation of the gradient vector of each weight and bias includes:

[0201] Step 32a: Calculate the gradient vector using the expressions of the gradient vectors of the weights and biases; wherein the expressions of the gradient vectors of the weights and biases are respectively:

[0202]

[0203] Where L represents the above loss function; represents the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer; represents the weight from the nth neuron in the input layer to the ith neuron in the hidden layer; i = 1, 2, ... K; K is the number of neurons in the hidden layer; n = 1, 2 ... D; D represents the dimension of the feature vector; m = 1, 2, ... M; M represents the number of neurons in the output layer; represents the rate of change from the weight of the mth neuron in the output layer to the i-th neuron in the hidden layer; represents the rate of change of the bias of the mth neuron in the output layer; Represents the output value of the i-th neuron in the hidden layer; represents the rate of change from the weight of the i-th neuron in the hidden layer to the n-th neuron in the input layer; represents the rate of change of the bias of the i-th neuron in the hidden layer; Y k represents the kth forward propagation or

[0204] It should be noted that in step 32a, when calculating When m=i and m≠i are considered, the corresponding back propagation method is different when m=i or m≠i, that is, different methods are needed to back propagate the gradient. For the specific method of back propagation of the gradient, please refer to the above expression of the gradient vector.

[0205] For some examples, please refer to Fig. 9 In step 33, the adaptive learning rate parameters are obtained, including:

[0206] Step 33a: Calculate the learning rate after the kth forward propagation;

[0207] Step 33b: Calculate the intermediate variables of each weight;

[0208] Step 33c: For the Y in the above expression of the gradient vector k Update, and then use the expressions of the gradient vectors of each weight and bias to calculate g (1)k+1 ;

[0209] Step 33d: Calculate a new learning rate, and the adaptive learning rate parameter includes the new learning rate.

[0210] In the above step 33a, the above learning rate The expression is:

[0211]

[0212] Among them, g k represents the kth forward propagation or or or D is a preset coefficient. D can be regarded as a preset learning rate. Those skilled in the art can determine the value of the preset coefficient D according to actual needs. For example, D can be set to 0.0001.

[0213] In the above step 33b, the expression of the intermediate variable of each weight can be:

[0214]

[0215] Among them, Y k Represents the value of each weight after the kth forward propagation; After the kth forward propagation or The intermediate variable.

[0216] It should be noted that the purpose of obtaining the "intermediate variables of each weight" in step 33b is to subsequently calculate the step length β of the weight update after the kth forward propagation. k .

[0217] In the above step 33c, first use For the above gradient vector expression, Y k Update, and then use the expression of the gradient vector of each weight and bias to calculate in, represents the kth forward propagation or or or The intermediate variable.

[0218] In the above step 33d, the new learning rate is calculated according to the following expression:

[0219]

[0220] Among them, the adaptive learning rate parameters include and

[0221] For some examples, please refer to Fig.10 In step 34, the weights and biases are iteratively updated by performing the kth back propagation according to the gradient vector and the adaptive learning rate parameter, including:

[0222] Step 34a: Calculate the value Y of each weight after the kth forward propagation k+1 ;

[0223] Step 34b: Use Y first k+1 For the above gradient vector expression, Y k Update, and then use the expressions of the gradient vectors of each weight and bias to calculate g k+1 ;

[0224] Step 34c: Calculate the step size for updating the weights after the kth forward propagation;

[0225] Step 34d: Calculate g k+2 .

[0226] In the above step 34a, Y can be calculated according to the following expression: k+1 :

[0227]

[0228] Among them, Y k+1 Represents the value of each weight after the k+1th forward propagation;

[0229] In the above step 34b, first use Y k+1 For the above gradient vector expression, Y k Update, that is, Y k Replace with Y k+1 ; Among them, g k+1 represents the k+1th forward propagation or or or

[0230] In the above step 34c, the calculation is performed according to the following expression:

[0231]

[0232] Among them, β k Indicates the step size of weight update after the kth forward propagation;

[0233] In the above step 34d, g is calculated according to the following expression: k+2 :

[0234] gk+2 =(-g k+1 )+(-g k )×β k ;

[0235] Among them, g k+2 It means after the k+2th forward propagation or or or It can be understood that the g obtained by performing the k-th back propagation in step 34 is k+1 and g k+2 They are respectively used as the initial values ​​of the corresponding parameters in the k+1th back propagation in step 34 and the initial values ​​of the corresponding parameters in the k+2th back propagation in step 34.

[0236] It should be noted that, since the adaptive learning rate parameter (such as the new learning rate) has been obtained in step 33, the adaptive learning rate parameter can be used to calculate the updated step size β in step 34. k ; That is to say, this application can complete the step length β according to the data adaptively k If the above weights and biases change slowly, the step length β k Should be relatively small. If the above weights and biases change quickly, the step size β k It should be relatively large to complete the parameter adaptation, which can avoid exceeding the optimal solution and thus improve the training speed and stability of the color recognition model.

[0237] In step 35, those skilled in the art can determine the preset number threshold or set the threshold according to actual needs. In the training process of the above-mentioned color recognition model, the color corresponding to each neuron in each output layer is determined by the training sample image. For example, in the case where the above-mentioned training feature vector can be composed of a training color feature vector and a training texture feature vector, if the size of the above-mentioned sub-pixel value interval is equal to 8, the training color feature vector is obtained by splicing the first feature vector and the second feature vector of the red channel, the green channel, the blue channel and the hue channel, then the dimensions of the first feature vector and the second feature vector are both 32, and the dimension of the training color feature vector is 32×8=256; and if the training texture feature vector is obtained by splicing the features of the 8 above-mentioned moments, then the dimension of the above-mentioned training feature vector is 128+128+8=264, then the number of neurons in the input layer is also 264, and each neuron in the input layer corresponds to each element in the above-mentioned training feature vector. Afterwards, the color recognition model outputs the confidence / probability of the predicted color (see step 31d above), and finally the color recognition model uses the color category with the highest confidence / probability as the predicted color, that is, it identifies different colors in the training sample image according to the score of the output layer. In short, the recognition method of the present application first obtains preset features (such as feature vectors composed of color features and / or texture features), and then realizes the recognition of complex colors through a machine learning method, and passes the area to be recognized (such as the feature vector above) into the trained color recognition model, and the color represented by the neuron with the highest score in the output layer is the corresponding color category predicted.

[0238] The recognition method of the present application mainly includes the following parts: color space conversion, color feature extraction, color similarity measurement, color matching and color recognition. Among them, color similarity measurement mainly refers to the confidence / probability of the color predicted by the output of the above-mentioned color recognition model (see the above-mentioned step 31d), and color matching mainly refers to the part of taking the color category with the largest confidence / probability as the predicted color.

[0239] It can be seen that in some embodiments, since the recognition method first establishes a high-dimensional feature vector through step S100, the feature vector can adapt to the influence of brightness changes and noise; by using the color recognition model in step S200 to learn the feature vector, a nonlinear structure is constructed, so it can adapt to changes in the external environment; in the color recognition process of the color recognition model, the minimum distance from the center is not used for color classification, so the method has better robustness. In summary, the method has good generalization performance, can adapt to environmental brightness changes, noise, etc., and improve the accuracy and stability of color recognition.

[0240] It can be seen that in some embodiments, the first feature vector of the recognition method is obtained based on the probability of the pixel values ​​in the red channel, the green channel, the blue channel and the hue channel being in each sub-pixel value interval, and the sub-pixel value interval is obtained by dividing the value range of the pixel values ​​in the red channel, the green channel, the blue channel and the hue channel, respectively, and the width of the sub-pixel value interval is greater than 1, that is, the barrel statistical information in the present application refers to the number of times or probability that each gray value appears in the corresponding sub-pixel value interval; that is, when the external environment changes to a certain extent, although a gray value in the image to be recognized may have changed, the gray value after the change is still likely to be in the original sub-pixel value interval, therefore, the recognition method of the present application can adapt to certain environmental changes and illumination changes, thereby improving the adaptability of the recognition method to illumination, environmental changes, etc. It can be seen that in some embodiments, since the adaptive learning rate parameter (such as the new learning rate) has been obtained in the above step 33, the adaptive learning rate parameter can be used in step 34 to calculate the updated step size β k ; That is to say, this application can complete the step length β according to the data adaptively k If the above weights and biases change slowly, the step length β k Should be relatively small. If the above weights and biases change quickly, the step size β k It should be relatively large to complete the parameter adaptation, which can avoid exceeding the optimal solution and thus improve the training speed and stability of the color recognition model.

[0241] It can be seen that in some embodiments, since the feature vector used in the present recognition method is composed of a color feature vector and a texture feature vector, and the color feature vector is composed of a first feature vector and a second feature vector, and the first feature vector and the second feature vector can represent colors from a variety of different color spaces, thereby enhancing the ability to represent mixed colors. For example, if the image contains complementary colors, the mean of the grayscale values ​​in its red channel, green channel and blue channel may be close to a single color, but the numerical difference in the hue channel in its HSV image is relatively large, and the color representation ability can be effectively enhanced by the color feature vector composed of the first feature vector and the second feature vector; in addition, since the first feature vector and the second feature vector only consider the statistics of the grayscale and ignore the relationship between the above-mentioned position and grayscale, the calculation of the texture feature vector not only considers the spatial position relationship, but also considers the grayscale information, and performs a correlation analysis on the two; that is, the texture feature vector describes the relationship between the position of the pixel point in the image and the grayscale of the pixel point, effectively enhancing the ability to express the mixed color, so the present recognition method can realize the recognition of complex backgrounds, mixed colors, etc., and meet the requirements for stable detection of complex colors.

[0242] The above is some description about the complex color recognition method. In some embodiments of the present application, a computer program product is also disclosed, including a computer program and / or instructions, which implement the recognition method as any one of the embodiments in this document when executed by a processor.

[0243] Some embodiments of the present application also disclose a computer-readable storage medium, including a program, which can be executed by a processor to implement an identification method as described in any embodiment of the present invention.

[0244] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0245] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. In addition, as understood by those skilled in the art, the principles of this article can be reflected in a computer program product on a computer-readable storage medium, which is pre-installed with a computer-readable program code. Any tangible, non-temporary computer-readable storage medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD to ROM, DVD, Blu Ray disks, etc.), flash memory and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions can also be stored in a computer-readable memory, which can instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory can form a manufactured product, including an implementation device that implements a specified function. Computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on a computer or other programmable device to generate a computer-implemented process, so that the instructions executed on a computer or other programmable device can provide steps for implementing a specified function.

[0246] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments will be included in the scope of this invention.

[0247] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0248] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. A complex color recognition method, characterized in that: include: Extracting a feature vector from the image to be identified, wherein the feature vector is a 1×n vector composed of n physical features; wherein the physical features include color features and / or texture features of the image to be identified; The feature vector is input into a trained color recognition model, and the color recognition model outputs the category and confidence of the color in the image to be recognized; or, the feature vector is normalized to obtain the standardized feature vector, and the standardized feature vector is input into a trained color recognition model, and the color recognition model outputs the confidence of the color in the image to be recognized; and the category of the color in the image to be recognized is determined according to the confidence.

2. The identification method according to claim 1, characterized in that: The step of extracting a feature vector from the image to be identified comprises: Extracting a color feature vector of the image to be identified; wherein the color feature vector is a 1×n1 feature vector composed of statistical information of the color of the red channel, the green channel, the blue channel of the image to be identified and / or the channel obtained by transforming the red channel, the green channel, and the blue channel of the image; Extracting a texture feature vector of the image to be identified, wherein the texture feature vector is a 1×n2 feature vector composed of the texture features, wherein the texture feature is a feature of a color moment calculated from the position information and color information of the image to be identified, and n=n1+n2; The feature vector is composed of the color feature vector, or the feature vector is composed of the color feature vector and the texture feature vector.

3. The identification method according to claim 2, characterized in that: The step of extracting the color feature vector of the image to be identified comprises: Performing HSV transformation on the image to be identified to obtain an HSV image; The pixel value ranges in the red channel, green channel, blue channel of the image to be identified and the hue channel of the HSV image are respectively divided into a plurality of sub-pixel value intervals; n1 is obtained based on the value obtained by dividing the size of the value range by the size of the sub-pixel value interval; The color feature vector is formed by statistical information of each sub-pixel value interval of the red channel, the green channel, the blue channel and the hue channel of the image to be identified.

4. The identification method according to claim 3, characterized in that: The color feature vector is formed by statistical information of each sub-pixel value interval of the red channel, the green channel, the blue channel and the hue channel of the image to be identified, including: Obtaining a first eigenvector of the red channel, the green channel, the blue channel and / or the hue channel; wherein the first eigenvector is obtained based on the probability of pixel values ​​in the red channel, the green channel, the blue channel and the hue channel being in each sub-pixel value interval; wherein the sub-pixel value interval is obtained by dividing the value range of the pixel values ​​in the red channel, the green channel, the blue channel and the hue channel respectively; wherein the width of the sub-pixel value interval is greater than 1; Obtaining a second eigenvector of the red channel, the green channel, the blue channel and / or the hue channel; wherein the second eigenvector is obtained based on the average values ​​of pixel values ​​in the red channel, the green channel, the blue channel and the hue channel that are within each of the sub-pixel value intervals; The color feature vector is obtained by concatenating the first feature vectors of each channel, or the color feature vector is obtained by concatenating the second feature vectors of each channel, or the color feature vector is obtained by concatenating the first feature vector of each channel and the second feature vector of each channel.

5. The identification method according to claim 2, characterized in that: The step of extracting the texture feature vector of the image to be identified comprises: Obtain multiple features of the moment; wherein the expression of the feature of the moment is: Among them, x, y and represents the abscissa, the average value of the abscissa, the ordinate and the average value of the ordinate of the pixel in the image to be identified, f(x, y) represents the gray value of the pixel, p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R here represent the maximum value of the abscissa and the maximum value of the ordinate, respectively; The texture feature vector is obtained by concatenating the features of a plurality of the moments.

6. The identification method according to claim 2, characterized in that: The step of extracting the texture feature vector of the image to be identified comprises: Obtain multiple features of the moment; wherein the expression of the feature of the moment is: Among them, the x, y and represents the abscissa, the average value of the abscissa, the ordinate and the average value of the ordinate of the pixel in the image to be identified, respectively; f(x, y) represents the gray value of the pixel in the image to be identified; p is equal to 0 or 1 or 2, q is equal to 0 or 1 or 2; C and R here represent the maximum value of the abscissa and the maximum value of the ordinate, respectively; The features of the moments are normalized using the following expression to obtain multiple normalized features η: p,q : The multiple normalized features η p,q The texture feature vector is obtained by splicing.

7. The identification method according to claim 1, characterized in that: The step of normalizing the feature vector to obtain the normalized feature vector includes: The feature vector is normalized using the following expression to obtain the normalized feature vector: Among them, the X i represents the ith eigenvector, X′ is the ith eigenvector after standardization, μ represents the mean of the ith eigenvector, σ is the standard deviation of the ith eigenvector, n is the dimension of the eigenvector, X i,j Represents the value of the eigenvector of the jth dimension of the i-th eigenvector.

8. The identification method according to claim 1, characterized in that: The color recognition model is a multi-layer perceptron, which includes an input layer, a hidden layer and an output layer; The number of neurons in the input layer of the multilayer perceptron is the same as the dimension of the feature vector, and the dimension of the feature vector is the sum of the dimensions of the vectors included in the feature vector; The number of neurons in the output layer of the multilayer perceptron is the same as the total number of color categories in the image to be recognized, and the colors represented by each neuron in the output layer are different.

9. The identification method according to claim 8, characterized in that: The color recognition model is trained through the following process: Obtain training feature vectors and corresponding annotation data of training sample images; Input the training feature vector into a color recognition model, the color recognition model predicts the color in the training sample image to obtain the category of the predicted color and the category probability corresponding to the predicted color; or, perform standardization on the training feature vector to obtain the standardized training feature vector, input the standardized training feature vector into the color recognition model, the color recognition model predicts the color in the training sample image to obtain the category of the predicted color and the confidence corresponding to the predicted color; The color recognition model is trained according to a loss function to obtain the trained color recognition model; the loss function is determined according to the confidence and the label value of the color category in the training sample image; the label value is obtained from the annotated data.

10. The identification method according to claim 9, characterized in that: The expression of the loss function is: Here, C represents the number of the training sample images; M represents the total number of color categories in the training sample images; represents the confidence of the i-th predicted color in the c-th training sample image; Represents the label value of the category of the i-th color in the c-th training sample image.

11. The identification method according to claim 10, characterized in that: The step of training the color recognition model according to the loss function to obtain the trained color recognition model includes: Step 30: Obtaining initial values ​​of weights and biases in the input layer, the hidden layer, and the output layer; Step 31: The color recognition model performs the kth forward propagation; after calculating the kth forward propagation The value of Step 32: Using the Calculate the gradient vector of each weight and the bias based on the value of ; Step 33: Get adaptive learning rate parameters; Step 34: performing a k-th back propagation according to the gradient vector and the adaptive learning rate parameter to iteratively update each of the weights and biases; Step 35: Repeat steps 30 to 34 until the number of back propagation exceeds a preset threshold, or the absolute value of the difference between the loss function in two adjacent back propagations is less than a set threshold, then stop the iterative update and use the multilayer perceptron at this time as the trained color recognition model.

12. The identification method according to claim 11, characterized in that: The color recognition model performs the kth forward propagation, including: Perform forward propagation from the input layer to the hidden layer; wherein the expression of the forward propagation from the input layer to the hidden layer is: in, is the weight from the nth neuron in the input layer to the ith neuron in the hidden layer, is the bias of the i-th neuron in the hidden layer; The y i Input to the nonlinear activation function to obtain Among them, the The expression is: Perform forward propagation from the hidden layer to the output layer; wherein the expression of the forward propagation from the hidden layer to the output layer is: Among them, the is the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer, is the bias of the hidden layer; The output of the output layer is processed by the Softmax function to obtain the Among them, the The expression is:

13. The identification method according to claim 12, characterized in that: The use of The numerical calculation of the gradient vector of each of the weights and the bias includes: The gradient vector is calculated using expressions of the gradient vectors of the weights and the biases; wherein the expressions of the gradient vectors of the weights and the biases are respectively: Wherein, L represents the loss function; represents the weight from the i-th neuron in the hidden layer to the m-th neuron in the output layer; represents the weight from the nth neuron in the input layer to the ith neuron in the hidden layer; i=1, 2, ... K; K is the number of neurons in the hidden layer; n=1, 2 ... D; D represents the dimension of the feature vector; m=1, 2, ... M; M represents the number of neurons in the output layer; represents the rate of change from the weight of the mth neuron in the output layer to the ith neuron in the hidden layer; represents the rate of change of the bias of the mth neuron in the output layer; represents the output value of the i-th neuron in the hidden layer; represents the rate of change from the weight of the i-th neuron in the hidden layer to the n-th neuron in the input layer; represents the rate of change of the bias of the i-th neuron in the hidden layer; the Y k Represents the k-th forward propagation after or 14. The identification method according to claim 13, characterized in that: The step of obtaining the adaptive learning rate parameter includes: The learning rate after the kth forward propagation is calculated according to the following expression Among them, the g k Represents the k-th forward propagation after or or or The D is a preset coefficient; The intermediate variables of each weight are calculated according to the following expression: Among them, the Y k represents the value of the weight after the k-th forward propagation; It means that after the kth forward propagation or The intermediate variable of First use the For the above gradient vector expression, Y k Update, and then use the expression of the gradient vector of each weight and the bias to calculate Among them, the Represents the k-th forward propagation after or or or The intermediate variable of The new learning rate is calculated according to the following expression: Wherein, the adaptive learning rate parameters include the and 15. The identification method according to claim 14, characterized in that: The step of performing a k-th back propagation according to the gradient vector and the adaptive learning rate parameter to iteratively update each of the weights and biases comprises: Calculated according to the following expression: Among them, the Y k+1 represents the value of each weight after the k-th forward propagation; First use the Y k+1 For the above gradient vector expression, Y k Update, and then use the expression of the gradient vector of each weight and the bias to calculate g k+1 ; wherein said g k+1 Represents the k+1th forward propagation after or or or Calculated according to the following expression: The β k represents the step size of updating the weight after the k-th forward propagation; Calculate g according to the following expression k+2 : g k+2 =(-g k+1 )+(-g k )×β k ; Among them, the g k+2 Represents the k+1th forward propagation after or or or 16. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the identification method according to any one of claims 1 to 15 is implemented.

17. A computer-readable storage medium, characterized in that: The invention comprises a program which can be executed by a processor to implement the identification method according to any one of claims 1 to 15.

Images