Image processing method, device, mobile terminal and readable storage medium

Abstract

The present invention provides an image processing method, device, mobile terminal and readable storage medium, which relate to the field of computer vision technology. The method comprises: acquiring a target image; filtering the target image through a filter to obtain differential information of the target image; constructing M target features of the target image according to the differential information, and obtaining M feature maps corresponding to the M target features respectively; wherein M is a positive integer; the pixels in the feature map correspond to the pixels in the target image; according to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image. The scheme of the present invention solves the problem in the prior art that the texture descriptor is not robust to grayscale changes caused by rotation, illumination, etc. of the image, which increases the difficulty of the image classification task.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an image processing method, device, mobile terminal and readable storage medium. Background Art

[0002] Acquiring texture features of images is an important task in computer vision. To achieve this goal, many efficient and powerful texture descriptors have been proposed in the field of image processing, which have accelerated the development of texture segmentation, texture classification, and scene recognition. Generally, the goal of basic texture descriptors is to distinguish images or objects by texture. However, these goals are affected by many factors. For example, when the image is disturbed by rotation, scale, and lighting, the difficulty of the classification task will increase significantly.

[0003] The classic local binary pattern (LBP) is a simple and effective texture descriptor. Its basic principle is to obtain the grayscale difference between the central pixel and the adjacent pixels, quantize each group of differences into binary codes, and use the frequency of occurrence of various binary codes in the image to form a histogram as a classification feature.

[0004] According to the basic principle of LBP, LBP is based on grayscale differential coding, which is not robust when the grayscale changes dramatically. Therefore, the texture descriptor based on LBP also has this problem. When the image illumination changes, or the foreground (or background) changes, the grayscale of the image will be reversed, and this change will interfere with the generation of binary codes, which will greatly reduce the classification accuracy of the image.

[0005] In summary, rotation and illumination are common factors affecting grayscale changes in natural images. However, existing texture descriptors are not robust to grayscale changes caused by rotation, illumination, etc., which makes image classification tasks more difficult. Summary of the invention

[0006] The purpose of the present invention is to provide an image processing method, device, mobile terminal and readable storage medium, which solves the problem in the prior art that the texture descriptor is not robust to grayscale changes caused by rotation, lighting, etc., which makes the image classification task more difficult.

[0007] To achieve the above object, an embodiment of the present invention provides an image processing method, comprising:

[0008] Get the target image;

[0009] Filtering the target image through a filter to obtain differential information of the target image;

[0010] According to the differential information, M target features of the target image are constructed, and M feature maps corresponding to the M target features are obtained respectively; wherein M is a positive integer; and the pixel points in the feature maps correspond to the pixel points in the target image;

[0011] According to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image.

[0012] Optionally, filtering the target image through a filter to obtain differential information of the target image includes:

[0013] The target image is filtered by the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; the differential structure information includes first-order differential structure information and second-order differential structure information;

[0014] According to the differential structure information, the differential information of the target image is obtained; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0015] Optionally, the target feature includes at least one of the following:

[0016] Shape index;

[0017] Mixed extreme value ratio;

[0018] Gradient amplitude;

[0019] Extreme difference;

[0020] Laplacian of Gaussian (Log) filtering characteristics;

[0021] Second-order derivative invariants.

[0022] Optionally, the jointly encoding the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image includes:

[0023] According to the M feature maps, the M target features are encoded to obtain binary code information corresponding to each pixel point in the M feature maps;

[0024] Selecting binary code information corresponding to pixel points with the same position coordinates in M ​​of the feature maps to form target binary code information; wherein a plurality of the target binary code information forms a target binary code information set;

[0025] According to the frequency of occurrence of each target binary code information in the target binary code information set, a feature histogram corresponding to the target image is obtained.

[0026] Optionally, encoding the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps includes:

[0027] According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps;

[0028] Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map;

[0029] According to the comparison result, the binary code information corresponding to each pixel point in each of the feature maps is determined, and the binary code information corresponding to each pixel point in M ​​feature maps is obtained respectively.

[0030] Optionally, determining the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result includes:

[0031] When the color value of the pixel point is greater than the corresponding color mean value, determining that the binary code information corresponding to the pixel point is the first preset information;

[0032] When the color value of the pixel point is less than or equal to the corresponding color mean value, it is determined that the binary code information corresponding to the pixel point is the second preset information.

[0033] To achieve the above object, an embodiment of the present invention provides an image processing device, comprising:

[0034] An image acquisition module, used for acquiring a target image;

[0035] An image filtering module, used for filtering the target image through a filter to obtain differential information of the target image;

[0036] A feature construction module, used to construct M target features of the target image according to the differential information, and obtain M feature maps corresponding to the M target features respectively; wherein M is a positive integer; and the pixel points in the feature map correspond to the pixel points in the target image;

[0037] A feature processing module is used to jointly encode the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image.

[0038] Optionally, the image filtering module includes:

[0039] A first filtering submodule, used for filtering the target image through the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; and the differential structure information includes first-order differential structure information and second-order differential structure information;

[0040] The second filtering submodule is used to obtain the differential information of the target image according to the differential structure information; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0041] Optionally, the target feature includes at least one of the following:

[0042] Shape index;

[0043] Mixed extreme value ratio;

[0044] Gradient amplitude;

[0045] Extreme difference;

[0046] Gaussian Laplace Log filter characteristics;

[0047] Second-order derivative invariants.

[0048] Optionally, the feature processing module includes:

[0049] A first processing submodule is used to encode the M target features according to the M feature maps, and obtain binary code information corresponding to each pixel point in the M feature maps;

[0050] The second processing submodule is used to select binary code information corresponding to pixel points with the same position coordinates in the M feature maps to form target binary code information; wherein a plurality of target binary code information forms a target binary code information set;

[0051] The third processing submodule is used to obtain a feature histogram corresponding to the target image according to the frequency of occurrence of each target binary code information in the target binary code information set.

[0052] Optionally, the first processing submodule includes:

[0053] A first determining unit, configured to determine a color mean value corresponding to each of the feature maps according to the color values ​​of the pixels on each of the feature maps;

[0054] An information comparison unit, used for comparing the color value of each pixel point on each of the characteristic maps with the color mean value corresponding to the characteristic map;

[0055] The second determination unit is used to determine the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result, and obtain the binary code information corresponding to each pixel point in the M feature maps respectively.

[0056] Optionally, the second determining unit includes:

[0057] A first encoding subunit, configured to determine, when the color value of the pixel point is greater than the corresponding color mean value, that the binary code information corresponding to the pixel point is the first preset information;

[0058] The second encoding subunit is used to determine that the binary code information corresponding to the pixel point is the second preset information when the color value of the pixel point is less than or equal to the corresponding color mean value.

[0059] To achieve the above object, an embodiment of the present invention provides a mobile terminal, including a processor and a transceiver, wherein the processor is used to:

[0060] Get the target image;

[0061] Filtering the target image through a filter to obtain differential information of the target image;

[0062] According to the differential information, M target features of the target image are constructed, and M feature maps corresponding to the M target features are obtained respectively; wherein M is a positive integer; and the pixel points in the feature maps correspond to the pixel points in the target image;

[0063] According to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image.

[0064] Optionally, when the processor is used to filter the target image through a filter to obtain differential information of the target image, it is specifically used to:

[0065] The target image is filtered by the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; the differential structure information includes first-order differential structure information and second-order differential structure information;

[0066] According to the differential structure information, the differential information of the target image is obtained; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0067] Optionally, the target feature includes at least one of the following:

[0068] Shape index;

[0069] Mixed extreme value ratio;

[0070] Gradient amplitude;

[0071] Extreme difference;

[0072] Gaussian Laplace Log filter characteristics;

[0073] Second-order derivative invariants.

[0074] Optionally, when the processor is used to jointly encode the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image, it is specifically used to:

[0075] According to the M feature maps, the M target features are encoded to obtain binary code information corresponding to each pixel point in the M feature maps;

[0076] Selecting binary code information corresponding to pixel points with the same position coordinates in M ​​of the feature maps to form target binary code information; wherein a plurality of the target binary code information forms a target binary code information set;

[0077] According to the frequency of occurrence of each target binary code information in the target binary code information set, a feature histogram corresponding to the target image is obtained.

[0078] Optionally, when the processor is used to encode the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps, the processor is specifically used to:

[0079] According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps;

[0080] Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map;

[0081] According to the comparison result, the binary code information corresponding to each pixel point in each of the feature maps is determined, and the binary code information corresponding to each pixel point in M ​​feature maps is obtained respectively.

[0082] Optionally, when the processor is used to determine the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result, it is specifically used to:

[0083] When the color value of the pixel point is greater than the corresponding color mean value, determining that the binary code information corresponding to the pixel point is the first preset information;

[0084] When the color value of the pixel point is less than or equal to the corresponding color mean value, it is determined that the binary code information corresponding to the pixel point is the second preset information.

[0085] To achieve the above objectives, an embodiment of the present invention provides a mobile terminal, including a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the image processing method as described above is implemented.

[0086] To achieve the above objective, an embodiment of the present invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the above-mentioned image processing method are implemented.

[0087] The beneficial effects of the above technical solution of the present invention are as follows:

[0088] The method of the embodiment of the present invention constructs multiple features of the image through the differential information of the image, so that these features have grayscale inversion and rotation invariance, and then obtains a grayscale inversion and rotation invariant histogram (GRH) by jointly encoding these features, so that the GRH feature has good classification performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is a flowchart of an image processing method according to an embodiment of the present invention;

[0090] Figure 2 A basic framework diagram of an image processing method according to another embodiment of the present invention;

[0091] Figure 3 is a structural diagram of a mobile terminal according to an embodiment of the present invention;

[0092] Figure 4 is a structural diagram of an image processing device according to an embodiment of the present invention;

[0093] Figure 5 This is a structural diagram of a mobile terminal according to another embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0095] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0096] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0097] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0098] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0099] like Figure 1 As shown, an image processing method according to an embodiment of the present invention includes:

[0100] Step 101, acquiring a target image;

[0101] Step 102: Filter the target image through a filter to obtain differential information of the target image.

[0102] In this step, the filter may be a first-order Gaussian derivative filter and a second-order Gaussian derivative filter, and the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0103] It should be noted that, since the target image itself contains rich structural information, various filters can controllably extract local or global structural information of the target image.

[0104] Step 103, constructing M target features of the target image according to the differential information, and obtaining M feature maps corresponding to the M target features respectively; wherein M is a positive integer; and the pixel points in the feature map correspond to the pixel points in the target image.

[0105] Step 104: jointly encode the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image.

[0106] In this embodiment, by selecting a variety of target features, the GRH can be made rotationally invariant and grayscale reversal invariant.

[0107] Specifically, we can first use the first-order Gaussian derivative and the second-order Gaussian derivative to obtain the differential structure information of the target image, and then use the extreme value response to obtain the differential information with rotation invariance (that is, the extreme value information of the target image in each direction of the filter), and then use the differential information of the target image to construct a variety of (for example, six) target features that are robust to grayscale changes; because the differential information cannot offset the grayscale inversion change, finally the dominant idea is used to jointly encode the six target features so that they can effectively resist sign changes.

[0108] Optionally, filtering the target image through a filter to obtain differential information of the target image includes:

[0109] The target image is filtered by the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; the differential structure information includes first-order differential structure information and second-order differential structure information;

[0110] According to the differential structure information, the differential information of the target image is obtained; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0111] In this embodiment, the image is filtered by a Gaussian derivative filter to obtain multiple (for example, six) features with image differential structure information, which is robust. The extreme value information in each direction (i.e., the first-order extreme value and the second-order extreme value) is obtained by the filter. Since these extreme values ​​are maximum values ​​or minimum values, the differential information obtained after filtering will not change with the rotation of the target image and is rotationally invariant.

[0112] Optionally, the target feature includes at least one of the following:

[0113] Shape index;

[0114] Mixed extreme value ratio;

[0115] Gradient amplitude;

[0116] Extreme difference;

[0117] Gaussian Laplace Log filter characteristics;

[0118] Second-order derivative invariants.

[0119] Specifically, in an optional embodiment of the present invention, the target feature can be obtained by the following method:

[0120] According to the formula Get the shape index;

[0121] According to the formula Get the mixed extreme value ratio;

[0122] According to the formula Get the gradient amplitude;

[0123] According to the formula Obtaining an extreme value difference; wherein the extreme value difference is the difference between the maximum value and the minimum value of the second-order image differential (i.e., the difference between the maximum value and the minimum value of the second-order extreme value information);

[0124] According to the formula log = |I xx +I yy |, get the Log filtering feature;

[0125] According to the formula Get the second-order derivative invariant;

[0126] Among them, s represents the shape index; r represents the mixed extreme value ratio; g represents the gradient amplitude; d represents the extreme value difference; log represents the Log filtering feature; e represents the second-order derivative invariant; I x Represents the first-order differential structure information along the x-axis; I xx Represents the second-order differential structure information along the x-axis; I y Represents the second-order differential structure information along the y-axis; I yy Represents the second-order differential structure information along the y-axis; I xy Represents the differential structure information obtained by differentiating along the x-axis and y-axis respectively; Indicates the maximum value of the first-order response (i.e., first-order extreme value information); Indicates the maximum value of the second-order response (i.e. the maximum value of the second-order extreme value information); Represents the minimum value of the second-order response (that is, the minimum value in the second-order extreme value information).

[0127] Optionally, the step of jointly encoding the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image comprises the following steps:

[0128] (i) Encoding the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps.

[0129] Optionally, this step may specifically include: determining the color mean corresponding to each feature map according to the color value of the pixel point on each feature map; comparing the color value of each pixel point on each feature map with the color mean corresponding to the feature map; determining the binary code information corresponding to each pixel point in each feature map according to the comparison result, and obtaining the binary code information corresponding to each pixel point in M ​​feature maps.

[0130] Here, the binary code information corresponding to a certain pixel point can be 0 or 1.

[0131] In this embodiment, the corresponding color mean value can be determined for each feature map, that is, a corresponding quantization threshold is used for each target feature, so that the GRH feature can have good classification performance.

[0132] (ii) selecting binary code information corresponding to pixel points with the same position coordinates in the M feature maps to form target binary code information; wherein a plurality of target binary code information forms a target binary code information set;

[0133] (iii) obtaining a feature histogram corresponding to the target image according to the frequency of occurrence of each target binary code information in the target binary code information set.

[0134] In this embodiment, the obtained feature histogram is a grayscale inversion and rotation invariant histogram.

[0135] Optionally, determining the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result includes:

[0136] When the color value of the pixel point is greater than the corresponding color mean value, determining that the binary code information corresponding to the pixel point is the first preset information;

[0137] When the color value of the pixel point is less than or equal to the corresponding color mean value, it is determined that the binary code information corresponding to the pixel point is the second preset information.

[0138] Here, the first preset information may be 1, and the second preset information may be 0. In this way, the binary code information corresponding to each pixel point in each feature spectrum may be determined according to the comparison result.

[0139] The solution provided in the embodiment of the present invention is specifically described below with examples.

[0140] Specifically, Figure 2 As shown, the image processing method of the embodiment of the present invention may include the following steps:

[0141] Step 1: Input image (i.e. obtain target image);

[0142] Step 2: Using a first-order Gaussian derivative filter and a second-order Gaussian derivative filter, the first-order differential structure information and the second-order differential structure information of the image are obtained;

[0143] It should be noted that the image itself contains rich structural information, and various filters can controllably extract local or global structural information of the image. Among them, the second-order Gaussian function is as follows:

[0144]

[0145] Among them, σ represents the standard deviation, also known as the scale; x and y represent the image coordinates of the Gaussian derivative to be calculated.

[0146] The first-order Gaussian derivative can be calculated by taking the derivative of the second-order Gaussian function in the x-direction and the y-direction. The expressions of the first-order Gaussian derivative in each direction are as follows:

[0147]

[0148] Among them, Gx represents the scale-normalized first-order Gaussian derivative along the x-axis; Gy represents the scale-normalized first-order Gaussian derivative along the y-axis; θ represents the direction of the first-order Gaussian derivative and the second-order Gaussian derivative.

[0149] The second-order Gaussian derivative can be calculated by taking the derivative of the second-order Gaussian function in the x-direction and the y-direction. The expressions of the second-order Gaussian derivative in each direction are as follows:

[0150]

[0151] Among them, G xx represents the scale-normalized second-order Gaussian derivative along the x-axis; G yy represents the scale-normalized second-order Gaussian derivative along the y-axis; G xy It means that the derivative is taken in the x direction first and then in the y direction; θ represents the direction of the first-order Gaussian derivative and the second-order Gaussian derivative.

[0152] The first-order differential structure information of the image is: I x =G x *I,I y =G y *I;

[0153] The second-order differential structure information of the image is: I xx =G xx *I,I yy =G yy *I;

[0154] Where I represents a digitized image and * represents convolution.

[0155] Step three: Calculate the first-order extreme response and the second-order extreme response of the image in each direction of the filter; that is, according to the differential structure information, obtain the differential information of the target image; wherein the differential information is the extreme information of the target image in each direction of the filter; wherein the extreme information includes first-order extreme information and second-order extreme information.

[0156] The maximum value of the first-order response (i.e., the first-order extreme value information) is:

[0157]

[0158] The maximum value of the second-order response (i.e. the maximum value of the second-order extreme value information) is:

[0159]

[0160] The minimum value of the second-order response (that is, the minimum value in the second-order extreme value information) is:

[0161]

[0162] Among them, I x Represents the first-order differential structure information along the x-axis; I xx Represents the second-order differential structure information along the x-axis; I y Represents the second-order differential structure information along the y-axis; I yy Represents the second-order differential structure information along the y-axis; I xy Represents the differential structure information obtained by differentiating along the x-axis and y-axis respectively; Indicates the maximum value of the first-order response; represents the maximum value of the second-order response; Represents the minimum value of the second-order response.

[0163] Since these values ​​(i.e. and ) is the maximum or minimum value, so the differential information obtained after filtering will not change with the rotation of the image and has rotation invariance.

[0164] Step 4: Based on the obtained image differential information (i.e., the maximum and minimum values ​​in the extreme value information), construct multiple (e.g., six) features (i.e., target features) that are robust to linear grayscale inversion changes, and the target features include at least one of the following: shape index, mixed extreme value ratio, gradient amplitude, extreme value difference, Log filter feature, and second-order derivative invariant;

[0165] Among them, the shape index s is a quantitative description of the local second-order curvature:

[0166]

[0167] The mixed extreme value ratio r can capture the image differential structure information, and the mixed extreme value ratio r can be obtained according to the following formula:

[0168]

[0169] The gradient magnitude g is the maximum value of the first-order image differential (that is, the maximum value of the first-order extreme value information):

[0170]

[0171] The extreme value difference d is the difference between the maximum and minimum values ​​of the second-order image differential (that is, the difference between the maximum and minimum values ​​of the second-order extreme value information):

[0172]

[0173] Log filter feature log:

[0174] log=|I xx +I yy |

[0175] Second-order derivative invariant e:

[0176]

[0177] Step 5: Jointly encode the six features based on the dominant idea; the basic idea of ​​the dominant encoding (i.e., joint encoding based on the dominant idea) is described as follows:

[0178] The mathematical model of linear grayscale change of image can be expressed as follows:

[0179] y=ax+b

[0180] To deal with b in the formula, it can be eliminated by addition, subtraction and differentiation, but the changes brought by a will be further retained, and the positive and negative changes of a will cause great interference to the encoding. To address this problem, dominant encoding is used to overcome the positive and negative changes in the grayscale value brought by a.

[0181] In dominant coding, first, a mean t (i.e., color mean) is calculated for the entire image (i.e., the feature map of the target image); then, the mean t is compared with the values ​​of all pixels (i.e., the color values ​​of the pixels on the feature map), as shown below:

[0182]

[0183] Among them, s1 represents the number of pixels in the image whose color value is greater than the color mean; n represents the number of pixels; p i represents the color value of the pixel; t represents the color mean.

[0184] Another quantity s2=n-s1 can also be defined, where S2 represents the number of pixels in the image whose color values ​​are less than or equal to the color mean.

[0185] As an optional embodiment, the encoding rules may be defined as follows:

[0186]

[0187] That is to say, it will belong to The pixel point p i The code is 1, otherwise, the code is 0, which means The pixel point p i Encoded as 0.

[0188] Among them, a i Represents the binary code of the pixel; p i Represents the i-th pixel; Represents the set of pixels in the image whose color values ​​are greater than the color mean; Represents the set of pixels in the image whose color values ​​are greater than the color mean.

[0189] In summary, pixels whose color values ​​are greater than the corresponding color mean can be encoded as 1 (i.e., the binary code information corresponding to the pixel is determined to be 1); pixels whose color values ​​are less than or equal to the corresponding color mean can be encoded as 0 (i.e., the binary code information corresponding to the pixel is determined to be 0).

[0190] It should be noted that since the number of pixels greater than t does not change with the change of the sign of a, it can be said that the dominant coding has grayscale reversal invariance.

[0191] Step 6: Combine the pixels at the same position in the six image differential features (i.e., the feature maps corresponding to the target features) to form a six-bit binary code (ie, target binary code information), the frequency of occurrence of the binary code in the image is counted to form a histogram, which is the grayscale reversal and rotation invariant histogram GRH.

[0192] In this step, binary code information corresponding to pixel points with the same position coordinates in M ​​(for example, M is 6) feature maps can be selected to form target binary code information; wherein, multiple target binary code information form a target binary code information set; then, according to the frequency of occurrence of each target binary code information in the target binary code information set, a feature histogram corresponding to the target image is obtained, that is, a grayscale reversal and rotation invariant histogram GRH.

[0193] In the embodiment of the present invention, multiple complementary features (i.e., target features) are added during the encoding process, and these features have both grayscale inversion and rotation invariance. In the process of jointly encoding multiple features, each feature adopts a corresponding quantization threshold, so that the GRH feature has good classification performance.

[0194] In one embodiment of the present invention, since the GRH feature has grayscale inversion invariance, in order to verify the performance of GRH, the used data set is subjected to grayscale inversion processing when compared with other methods.

[0195] As shown in Table 1 below, the classification accuracy (in %) of multi-scale GRH with σ=1, 2 and 3 and other methods under linear grayscale inversion is:

[0196]

[0197]

[0198] Among them, LTP is local ternary pattern (LTP); CLBP is complete local binary pattern (CLBP); CLBC is complete local binary count (CLBC); LETRIST is local encoded transform feature histogram (LETRIST); jcLSFP is joint coding of local space-frequency pattern (jcLSFP); SIFT is scale-invariant feature transform (SIFT); LDZP is local directional ZigZag pattern (LDZP); LOOP is local optimal-oriented pattern (LOOP); GLBP is gradient local binary patterns (GLBP); LGP is local gradient patterns (LGP); NRLBP is non-redundant local binary patterns (NRLBP). binary patterns, NRLBP); SLGP is the sorted local gradient pattern (sorted local gradient pattern, SLGP).

[0199] In Table 1 above, the Outex database is used, and two sub-databases, TC10 and TC12, are selected for experiments. The TC10 sub-database is used to test rotation invariance. It contains 24 types of texture images imaged under the illumination condition "inca", with 20 images in each category and 9 rotation angles (0±, 5±, 10±, 15±, 30±, 45±, 60±, 75± and 90±). Among them, 480 (24×20) images with a rotation angle of 0± are used as training sets, and 3840 (24×20×8) images with the remaining 8 rotation angles are used as test sets. The TC12 sub-database is used to test illumination invariance. Its training set is the same as TC10, and the test set is 4320 (24×20×9) images imaged under the illumination conditions "tl84" or "horizon". The CUReT database contains 61 types of textures, each with 92 images, imaged under different viewing angles and illumination directions. 46 images are randomly selected from each category as the training set, and the remaining images are used as the test set. The KTH-TIPS database contains 10 types of textures, each with 81 images, showing different scales, illumination and posture changes. 40 images are randomly selected from each category as the training set, and the rest are used as the test set. Since the CUReT and KTH-TIPS databases do not have a fixed division between training and test sets, the embodiment of the present invention conducts multiple classification experiments and then calculates the average classification accuracy.

[0200] Table 1 compares the classification accuracy of different methods on the linear grayscale inversion dataset. Since the parameters of variant descriptors such as LBP contain radius, in the experiment, we conducted experiments on LBP, LTP, CLBP, LGP, GLBP, and NRLBP under the conditions of r=1, r=2, r=3, and the combination of the three radii, and compared the data of the best performance of these descriptors under the four conditions with the data of the GRH method in Tables 1 and 2.

[0201] It can be seen from Table 1 that the inversion of grayscale will cause the classification performance of the descriptor based on the traditional LBP to drop sharply. The reason is that the huge change of grayscale value will affect the grayscale difference between the neighboring pixels and the central pixel, so that the binary code generated based on the grayscale difference is no longer the same as the binary code of the original image. When the histogram is used for comparison, the distance between the image and the original class will become larger, which will eventually cause image classification errors.

[0202] For methods without rotation invariance (such as NRLBP, etc.), the performance is not ideal in the TC10 database with rotation changes. In terms of overcoming the problem of illumination changes, GLBP and SLGP have better performance on the TC12 database than other methods. The method GRH in the embodiment of the present invention has relatively ideal performance on both the TC10 database with rotation changes and the TC12 database with illumination changes, indicating that the features of GRH can effectively resist the changes caused by image rotation and illumination.

[0203] As shown in Table 2 below, the classification accuracy (%) of multi-scale GRH with σ=1, 2 and 3 and other methods under nonlinear grayscale inversion:

[0204]

[0205] For nonlinear illumination changes, we have also preliminarily verified the performance of GRH. By comparing the performance of GRH in Table 1 and Table 2, it can be seen that the performance degradation of GRH under various databases is not as serious as other methods, indicating that GRH is also robust to nonlinear illumination changes.

[0206] As shown in Tables 1 and 2 above, the GRH features used in the embodiments of the present invention can achieve good classification performance under linear and nonlinear grayscale inversion conditions; wherein the scale σ of GRH is a combination of multiple scales of 1, 2, and 3.

[0207] As shown in Table 3 below, the classification accuracy (%) of GRH of different scales under linear grayscale inversion is:

[0208]

[0209] As shown in Table 3 above, the experiment tested the impact of scale on GRH performance. As can be seen from Table 3: Under single scale, σ = 2 has the best performance, and as the scale increases, the performance of GRH decreases; for joint scale, the more joint scales, the better the performance, but when the joint scale is σ = 4, the performance of GRH is not improved much, and the performance under some databases decreases. Therefore, when comparing with other methods, we use multi-scale GRH with σ of 1, 2, and 3.

[0210] As shown in Table 4 below, the classification accuracy (%) of various features in the multi-scale GRH with σ=1 and 2 under linear grayscale inversion:

[0211]

[0212] As shown in Table 4 above, the embodiment of the present invention adopts a total of six image differential features. To verify the effectiveness of each feature, we conduct a linear superposition experiment on the features. In the experiment in Table 4, the scale σ of GRH is a combination of 1 and 2. It can be seen from Table 4 that: with the addition of a new feature, the performance of GRH in all databases is improved, and this improvement is particularly obvious when the performance of GRH is low. When the number of features reaches more than five, the performance improvement of GRH is no longer obvious, and the performance under some databases has slightly decreased, indicating that the features of GRH have begun to become redundant. Therefore, we finally selected six features as the underlying features of GRH. Experiments show that GRH performs well in classification tasks under rotation and illumination changes.

[0213] The method of the embodiment of the present invention constructs multiple features of the image through the differential information of the image, so that these features have grayscale inversion and rotation invariance, and then obtains GRH by jointly encoding these features, so that the GRH feature has good classification performance.

[0214] like Figure 3 As shown, a mobile terminal 300 according to an embodiment of the present invention includes a processor 310 and a transceiver 320, wherein the processor 310 is configured to:

[0215] Get the target image;

[0216] Filtering the target image through a filter to obtain differential information of the target image;

[0217] According to the differential information, M target features of the target image are constructed, and M feature maps corresponding to the M target features are obtained respectively; wherein M is a positive integer; and the pixel points in the feature maps correspond to the pixel points in the target image;

[0218] According to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image.

[0219] Optionally, when the processor 310 is used to filter the target image through a filter to obtain differential information of the target image, it is specifically used to:

[0220] The target image is filtered by the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; the differential structure information includes first-order differential structure information and second-order differential structure information;

[0221] According to the differential structure information, the differential information of the target image is obtained; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0222] Optionally, the target feature includes at least one of the following:

[0223] Shape index;

[0224] Mixed extreme value ratio;

[0225] Gradient amplitude;

[0226] Extreme difference;

[0227] Gaussian Laplace Log filter characteristics;

[0228] Second-order derivative invariants.

[0229] Optionally, when the processor 310 is used to jointly encode the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image, it is specifically used to:

[0230] According to the M feature maps, the M target features are encoded to obtain binary code information corresponding to each pixel point in the M feature maps;

[0231] Selecting binary code information corresponding to pixel points with the same position coordinates in M ​​of the feature maps to form target binary code information; wherein a plurality of the target binary code information forms a target binary code information set;

[0232] According to the frequency of occurrence of each target binary code information in the target binary code information set, a feature histogram corresponding to the target image is obtained.

[0233] Optionally, when the processor 310 is used to encode the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps, the processor 310 is specifically used to:

[0234] According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps;

[0235] Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map;

[0236] According to the comparison result, the binary code information corresponding to each pixel point in each of the feature maps is determined, and the binary code information corresponding to each pixel point in M ​​feature maps is obtained respectively.

[0237] Optionally, when the processor 310 is used to determine the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result, it is specifically used to:

[0238] When the color value of the pixel point is greater than the corresponding color mean value, determining that the binary code information corresponding to the pixel point is the first preset information;

[0239] When the color value of the pixel point is less than or equal to the corresponding color mean value, it is determined that the binary code information corresponding to the pixel point is the second preset information.

[0240] The mobile terminal of this embodiment constructs multiple features of the image through differential information of the image, so that these features have grayscale inversion and rotation invariance, and then obtains GRH by jointly encoding these features, so that the GRH feature has good classification performance.

[0241] like Figure 4 As shown, an embodiment of the present invention provides an image processing device, comprising:

[0242] An image acquisition module 410 is used to acquire a target image;

[0243] An image filtering module 420, configured to filter the target image through a filter to obtain differential information of the target image;

[0244] A feature construction module 430 is used to construct M target features of the target image according to the differential information, and obtain M feature maps corresponding to the M target features respectively; wherein M is a positive integer; and the pixel points in the feature map correspond to the pixel points in the target image;

[0245] The feature processing module 440 is used to jointly encode the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image.

[0246] Optionally, the image filtering module 420 includes:

[0247] A first filtering submodule, used for filtering the target image through the filter to obtain differential structure information of the target image; wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; and the differential structure information includes first-order differential structure information and second-order differential structure information;

[0248] The second filtering submodule is used to obtain the differential information of the target image according to the differential structure information; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

[0249] Optionally, the target feature includes at least one of the following:

[0250] Shape index;

[0251] Mixed extreme value ratio;

[0252] Gradient amplitude;

[0253] Extreme difference;

[0254] Gaussian Laplace Log filter characteristics;

[0255] Second-order derivative invariants.

[0256] Optionally, the feature processing module 440 includes:

[0257] A first processing submodule is used to encode the M target features according to the M feature maps, and obtain binary code information corresponding to each pixel point in the M feature maps;

[0258] The second processing submodule is used to select binary code information corresponding to pixel points with the same position coordinates in the M feature maps to form target binary code information; wherein a plurality of target binary code information forms a target binary code information set;

[0259] The third processing submodule is used to obtain a feature histogram corresponding to the target image according to the frequency of occurrence of each target binary code information in the target binary code information set.

[0260] Optionally, the first processing submodule includes:

[0261] A first determining unit, configured to determine a color mean value corresponding to each of the feature maps according to the color values ​​of the pixels on each of the feature maps;

[0262] An information comparison unit, used for comparing the color value of each pixel point on each of the characteristic maps with the color mean value corresponding to the characteristic map;

[0263] The second determination unit is used to determine the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result, and obtain the binary code information corresponding to each pixel point in the M feature maps respectively.

[0264] Optionally, the second determining unit includes:

[0265] A first encoding subunit, configured to determine, when the color value of the pixel point is greater than the corresponding color mean value, that the binary code information corresponding to the pixel point is the first preset information;

[0266] The second encoding subunit is used to determine that the binary code information corresponding to the pixel point is the second preset information when the color value of the pixel point is less than or equal to the corresponding color mean value.

[0267] The image processing device of this embodiment constructs multiple features of the image through differential information of the image, so that these features have grayscale inversion and rotation invariance, and then obtains GRH by jointly encoding these features, so that the GRH feature has good classification performance.

[0268] A mobile terminal according to another embodiment of the present invention, Figure 5 As shown, it includes a transceiver 510, a processor 500, a memory 520, and a program or instruction stored in the memory 520 and executable on the processor 500; when the processor 500 executes the program or instruction, the above-mentioned application is implemented. Image processing method.

[0269] The transceiver 510 is used to receive and send data under the control of the processor 500 .

[0270] Among them, Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 500 and various circuits of memory represented by memory 520 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 510 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 530 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0271] The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 can store data used by the processor 500 when performing operations.

[0272] A computer-readable storage medium according to an embodiment of the present invention stores a program or instruction thereon, and when the program or instruction is executed by a processor, the steps in the image processing method described above are implemented, and the same technical effect can be achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0273] It should be further explained that the terminals described in this specification include but are not limited to smart phones, tablet computers, etc., and many of the functional components described are called modules in order to more particularly emphasize the independence of their implementation methods.

[0274] In the embodiment of the present invention, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0275] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0276] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0277] The above exemplary embodiments are described with reference to the accompanying drawings, and many different forms and embodiments are feasible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be constructed as a limitation of the exemplary embodiments proposed herein. More specifically, these exemplary embodiments are provided so that the present invention will be perfect and complete, and the scope of the present invention will be conveyed to those who are familiar with the technology. In these figures, the component sizes and relative sizes may be exaggerated for clarity. The terms used here are only based on the purpose of describing specific exemplary embodiments and are not intended to be limiting. As used herein, unless the text clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to include these multiple forms. It will be further understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of the features, integers, steps, operations, components and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, components, components and / or their groups. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0278] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Get the target image; Filtering the target image through a filter to obtain differential information of the target image; According to the differential information, M target features of the target image are constructed, and M feature maps corresponding to the M target features are obtained respectively; wherein M is a positive integer; and the pixel points in the feature maps correspond to the pixel points in the target image; According to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image; The step of jointly encoding the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image includes: According to the M feature maps, the M target features are encoded to obtain binary code information corresponding to each pixel point in the M feature maps; Selecting binary code information corresponding to pixel points with the same position coordinates in M ​​of the feature maps to form target binary code information; wherein a plurality of the target binary code information forms a target binary code information set; Obtaining a feature histogram corresponding to the target image according to the frequency of occurrence of each target binary code information in the target binary code information set; The encoding of the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps includes: According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps; Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map; According to the comparison result, the binary code information corresponding to each pixel point in each of the feature maps is determined, and the binary code information corresponding to each pixel point in M ​​feature maps is obtained respectively.

2. The image processing method according to claim 1, characterized in that: The filtering the target image through a filter to obtain differential information of the target image includes: Filtering the target image by the filter to obtain differential structure information of the target image, wherein the filter includes a first-order Gaussian derivative filter and a second-order Gaussian derivative filter; the differential structure information includes first-order differential structure information and second-order differential structure information; According to the differential structure information, the differential information of the target image is obtained; wherein the differential information is the extreme value information of the target image in each direction of the filter; wherein the extreme value information includes first-order extreme value information and second-order extreme value information.

3. The image processing method according to claim 2, characterized in that: The target feature includes at least one of the following: Shape index; Mixed extreme value ratio; Gradient amplitude; Extreme difference; Gaussian Laplace Log filter characteristics; Second-order derivative invariants; Among them, the Log filtering feature is based on the formula get; The second-order derivative invariant is given by the formula get; Wherein, log represents the Log filtering feature, I xx represents the second-order differential structure information along the x-axis, I yy represents the second-order differential structure information along the y-axis, represents the second-order derivative invariant, I xy It represents the differential structure information obtained by differentiating along the x-axis and y-axis respectively.

4. The image processing method according to claim 1, characterized in that: Determining the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result includes: When the color value of the pixel point is greater than the corresponding color mean value, determining that the binary code information corresponding to the pixel point is the first preset information; When the color value of the pixel point is less than or equal to the corresponding color mean value, it is determined that the binary code information corresponding to the pixel point is the second preset information.

5. An image processing device, characterized in that: include: An image acquisition module, used for acquiring a target image; An image filtering module, used for filtering the target image through a filter to obtain differential information of the target image; A feature construction module, used to construct M target features of the target image according to the differential information, and obtain M feature maps corresponding to the M target features respectively; wherein M is a positive integer; and the pixel points in the feature map correspond to the pixel points in the target image; A feature processing module, used for jointly encoding the M target features according to the M feature maps to obtain a feature histogram corresponding to the target image; Wherein, the feature processing module includes: A first processing submodule is used to encode the M target features according to the M feature maps, and obtain binary code information corresponding to each pixel point in the M feature maps; The second processing submodule is used to select binary code information corresponding to pixel points with the same position coordinates in the M feature maps to form target binary code information; wherein a plurality of target binary code information forms a target binary code information set; A third processing submodule, configured to obtain a feature histogram corresponding to the target image according to the frequency of occurrence of each target binary code information in the target binary code information set; Wherein, the first processing submodule includes: A first determining unit, configured to determine a color mean value corresponding to each of the feature maps according to the color values ​​of the pixels on each of the feature maps; An information comparison unit, used for comparing the color value of each pixel point on each of the characteristic maps with the color mean value corresponding to the characteristic map; The second determination unit is used to determine the binary code information corresponding to each pixel point in each of the feature maps according to the comparison result, and obtain the binary code information corresponding to each pixel point in the M feature maps respectively.

6. A mobile terminal, characterized in that: include: processor; The processor is used to: Get the target image; Filtering the target image through a filter to obtain differential information of the target image; According to the differential information, M target features of the target image are constructed, and M feature maps corresponding to the M target features are obtained respectively; wherein M is a positive integer; and the pixel points in the feature maps correspond to the pixel points in the target image; According to the M feature maps, the M target features are jointly encoded to obtain a feature histogram corresponding to the target image; Wherein, when the processor is used to encode the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps, the processor is specifically used to: According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps; Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map; According to the comparison result, determine the binary code information corresponding to each pixel point in each of the feature maps, and obtain the binary code information corresponding to each pixel point in M ​​feature maps respectively; Wherein, when the processor is used to encode the M target features according to the M feature maps to obtain binary code information corresponding to each pixel point in the M feature maps, the processor is specifically used to: According to the color values ​​of the pixels on each of the characteristic maps, respectively determine the color mean corresponding to each of the characteristic maps; Compare the color value of each pixel on each of the characteristic maps with the color mean value corresponding to the characteristic map; According to the comparison result, the binary code information corresponding to each pixel point in each of the feature maps is determined, and the binary code information corresponding to each pixel point in M ​​feature maps is obtained respectively.

7. A mobile terminal, comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the image processing method according to any one of claims 1 to 4 when executing the program or instruction.

8. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps in the image processing method according to any one of claims 1 to 4 are implemented.

Images