Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, an electronic device and a storage medium, wherein the method comprises: acquiring image features of multiple images of a same object respectively; determining a weight coefficient corresponding to each image feature according to the image features of the images; and performing feature fusion processing on the image features of the multiple images based on the weight coefficients of the image features, to obtain fused features of the multiple images. The present disclosure can improve the accuracy of the fused features.

Description

[0001] This application is a divisional application of Chinese patent application filed on March 25, 2019, with application number 201910228716.X and title "Image Processing Method and Apparatus, Electronic Device and Storage Medium". Technical Field

[0002] This disclosure relates to the field of computer vision, and in particular to an image processing method and apparatus, electronic device and storage medium. Background Technology

[0003] Feature fusion is a crucial issue in computer vision and intelligent video surveillance. For example, facial feature fusion has significant applications in many fields, such as facial recognition systems. Currently, existing technologies typically use the average of features from multiple frames as the fused feature. While simple, this method suffers from poor performance, particularly in its poor robustness to outliers. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, electronic device, and storage medium for improving the accuracy of fusion features.

[0005] According to one aspect of this disclosure, an image processing method is provided, comprising:

[0006] Obtain image features from multiple images of the same object;

[0007] Based on the image features of each image, determine the weight coefficients that correspond one-to-one with each of the image features;

[0008] Based on the weight coefficients of each image feature, feature fusion processing is performed on the image features of the multiple images to obtain the fused features of the multiple images.

[0009] In some possible implementations, determining the weight coefficients corresponding one-to-one with each of the image features based on the image features of each image includes:

[0010] An image feature matrix is ​​formed based on the image features of each image;

[0011] Perform feature fitting processing on the image feature matrix to obtain the first weight matrix;

[0012] The weight coefficients corresponding to each image feature are determined based on the first weight matrix.

[0013] In some possible implementations, performing feature fitting processing on the image feature matrix to obtain a first weight matrix includes:

[0014] The image feature matrix is ​​subjected to feature fitting processing using a regularized linear least squares estimation algorithm, and the first weight matrix is ​​obtained when the preset objective function is minimized.

[0015] In some possible implementations, determining the weight coefficients corresponding to each image feature based on the first weight matrix includes:

[0016] Each first weight coefficient included in the first weight matrix is ​​determined as the weight coefficient corresponding to each image feature; or

[0017] Perform a first optimization process on the first weight matrix, and determine each first weight coefficient included in the optimized first weight matrix as the weight coefficient corresponding to each image feature.

[0018] In some possible implementations, performing the first optimization process on the first weight matrix includes:

[0019] Based on the first weight coefficients of each image feature included in the first weight matrix, the fitted image features of each image are determined, wherein the fitted image features are the product of the image features and the corresponding first weight coefficients.

[0020] Using the first error between the image features of each image and the fitted image features, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix;

[0021] In response to the first condition being met by the difference between the first weight matrix and the first optimized weight matrix, the first optimized weight matrix is ​​determined as the optimized first weight matrix.

[0022] If the difference between the first weight matrix and the first optimized weight matrix does not satisfy the first condition, new fitted image features are obtained using the first optimized weight matrix. The first optimization process is repeated based on the new fitted image features until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix satisfies the first condition. The k-th optimized weight matrix is ​​then determined as the optimized first weight matrix, where k is a positive integer greater than 1.

[0023] In some possible implementations, the first optimization process of the first weight matrix, utilizing the first error between the image features of each image and the fitted image features, includes:

[0024] The first error between the image features and the fitted image features is obtained by summing the squares of the differences between the corresponding elements in each image feature and the fitted image features.

[0025] The second weighting coefficients for each image feature are obtained based on each of the first errors;

[0026] Based on the second weight coefficients of each image, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix corresponding to the first weight matrix.

[0027] In some possible implementations, obtaining the second weighting coefficients for each image feature based on each of the first errors includes:

[0028] Using a first method, second weighting coefficients for each image feature are obtained based on each of the first errors, wherein the expression for the first method is:

[0029]

[0030] Among them, w i e is the second weighting coefficient for the i-th image. i Let represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer between 1 and N, N is the number of image features, and k = 1.345σ, where σ is the error e. i The standard deviation.

[0031] In some possible implementations, determining the weight coefficients corresponding one-to-one with each of the image features based on the image features of each image further includes:

[0032] An image feature matrix is ​​formed based on the image features of each image;

[0033] Median filtering is performed on the image feature matrix to obtain the median feature matrix;

[0034] The weight coefficients corresponding to each image feature are determined based on the median feature matrix.

[0035] In some possible implementations, performing median filtering on the image feature matrix to obtain a median feature matrix includes:

[0036] Determine the median value of each image feature in the image feature matrix for the same position;

[0037] The median feature matrix is ​​obtained based on the median value of the elements at each position.

[0038] In some possible implementations, determining the weight coefficients corresponding to each image feature based on the median feature matrix includes:

[0039] Obtain the second error between each image feature and the median feature matrix;

[0040] In response to the second error between the image feature and the median feature matrix satisfying the second condition, the weight coefficient of the image feature is configured as the first weight. In response to the second error between the image feature and the median feature matrix not satisfying the second condition, the weight coefficient of the image feature is determined using the second method.

[0041] In some possible implementations, the expression for the second method is:

[0042]

[0043] θ h =1 / e h ;

[0044] Among them, b h Let e ​​be the weight coefficient of the h-th image determined by the second method. h The second error between the image features of the h-th image and the median feature matrix is ​​denoted as h, where h is an integer value from 1 to N, and N represents the number of images.

[0045] In some possible implementations, the second condition is:

[0046] e h >K·MADN;

[0047] MADN = median([e1,e2,...e...) N ]) / 0.675;

[0048] Among them, e h Let be the second error between the image features of the h-th image and the median feature matrix, where h is an integer value from 1 to N, N represents the number of images, K is the judgment threshold, and median represents the median filtering function.

[0049] In some possible implementations, performing feature fusion processing on the image features of the plurality of images based on the weight coefficients of each of the image features to obtain fused features of the plurality of images includes:

[0050] The fused features are obtained by summing the products of each image feature and its corresponding weight coefficient.

[0051] In some possible implementations, the method further includes:

[0052] The same object is identified using the fused features.

[0053] In some possible implementations, before determining the weight coefficients corresponding to each image feature based on the image features of each image, the method further includes:

[0054] Obtain information on the selection mode for obtaining the weight coefficients;

[0055] The acquisition mode of the weight coefficients is determined based on the selection information;

[0056] Based on the determined acquisition mode of the weight coefficients, the weight coefficients corresponding to each image feature are determined according to the image features of each image.

[0057] The weight coefficients are obtained through two methods: feature fitting and median filtering.

[0058] According to a second aspect of this disclosure, an image processing apparatus is provided, comprising:

[0059] The acquisition module is used to acquire image features from multiple images of the same object.

[0060] The determining module is used to determine the weight coefficients corresponding one-to-one with each of the image features based on the image features of each image;

[0061] The fusion module is used to perform feature fusion processing on the image features of the plurality of images based on the weight coefficients of each of the image features, so as to obtain the fused features of the plurality of images.

[0062] In some possible implementations, the determining module includes:

[0063] The first establishing unit is used to form an image feature matrix based on the image features of each image;

[0064] A fitting unit is used to perform feature fitting processing on the image feature matrix to obtain a first weight matrix;

[0065] The first determining unit is used to determine the weight coefficients corresponding to each image feature based on the first weight matrix.

[0066] In some possible implementations, the fitting unit is further configured to perform feature fitting processing on the image feature matrix using a regularized linear least squares estimation algorithm, and obtain the first weight matrix when the preset objective function is minimized.

[0067] In some possible implementations, the determining module further includes an optimization unit for performing a first optimization process on the first weight matrix;

[0068] The first determining unit is further configured to determine each first weight coefficient included in the first weight matrix as the weight coefficient corresponding to each image feature; or to determine each first weight coefficient included in the optimized first weight matrix as the weight coefficient corresponding to each image feature.

[0069] In some possible implementations, the optimization unit is further configured to determine the fitted image features of each image based on the first weight coefficients of each image feature included in the first weight matrix.

[0070] Using the first error between the image features of each image and the fitted image features, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix;

[0071] In response to the first condition being met by the difference between the first weight matrix and the first optimized weight matrix, the first optimized weight matrix is ​​determined as the optimized first weight matrix.

[0072] In response to the first condition not being met by the difference between the first weight matrix and the first optimized weight matrix, new fitting image features are obtained using the first optimized weight matrix. The first optimization process is repeated based on the new fitting image features until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix meets the first condition. The k-th optimized weight matrix is ​​then determined as the optimized first weight matrix, where k is a positive integer greater than 1.

[0073] Wherein, the fitted image feature is the product of the image feature and the corresponding first weight coefficient.

[0074] In some possible implementations, the optimization unit is further configured to obtain a first error between the image features and the fitted image features based on the sum of squares of the differences between corresponding elements in each image feature and the fitted image features;

[0075] The second weighting coefficients for each image feature are obtained based on each of the first errors;

[0076] Based on the second weight coefficients of each image, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix corresponding to the first weight matrix.

[0077] In some possible implementations, the optimization unit is further configured to obtain second weight coefficients for each image feature based on each of the first errors using a first method, wherein the expression for the first method is:

[0078]

[0079] Among them, w ie is the second weighting coefficient for the i-th image. i Let represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer between 1 and N, N is the number of image features, and k = 1.345σ, where σ is the error e. i The standard deviation.

[0080] In some possible implementations, the determining module further includes:

[0081] The second establishing unit is used to form an image feature matrix based on the image features of each image;

[0082] A filtering unit is used to perform median filtering on the image feature matrix to obtain a median feature matrix;

[0083] The second determining unit is used to determine the weight coefficients corresponding to each image feature based on the median feature matrix.

[0084] In some possible implementations, the filtering unit is also used to determine the median value of each image feature in the image feature matrix for the same position;

[0085] The median feature matrix is ​​obtained based on the median value of the elements at each position.

[0086] In some possible implementations, the second determining unit is further configured to obtain a second error between each image feature and the median feature matrix;

[0087] In response to the second error between the image feature and the median feature matrix satisfying the second condition, the weight coefficient of the image feature is configured as the first weight. In response to the second error between the image feature and the median feature matrix not satisfying the second condition, the weight coefficient of the image feature is determined using the second method.

[0088] In some possible implementations, the expression for the second method is:

[0089]

[0090] θ h =1 / e h ;

[0091] Among them, b h Let e ​​be the weight coefficient of the h-th image determined by the second method. h The second error between the image features of the h-th image and the median feature matrix is ​​denoted as h, where h is an integer value from 1 to N, and N represents the number of images.

[0092] In some possible implementations, the second condition is:

[0093] e h >K·MADN;

[0094] MADN = median([e1,e2,...e...) N ]) / 0.675;

[0095] Among them, e h Let be the second error between the image features of the h-th image and the median feature matrix, where h is an integer value from 1 to N, N represents the number of images, K is the judgment threshold, and median represents the median filtering function.

[0096] In some possible implementations, the fusion module is further configured to obtain the fused features by summing the products of each image feature and its corresponding weight coefficient.

[0097] In some possible implementations, the device further includes an identification module for performing the identification operation of the same object using the fusion features.

[0098] In some possible implementations, the apparatus further includes a mode determination module for selecting information on the acquisition mode of the weight coefficients, and determining the acquisition mode of the weight coefficients based on the selection information. The acquisition mode of the weight coefficients includes acquiring the weight coefficients by feature fitting and acquiring the weight coefficients by median filtering.

[0099] The determining module is further configured to, based on the determined acquisition mode of the weight coefficients, perform the step of determining the weight coefficients corresponding to each image feature according to the image features of each image.

[0100] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0101] processor;

[0102] Memory used to store processor-executable instructions;

[0103] The processor is configured to execute the method described in any one of the first aspects.

[0104] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the method described in any one of the first aspects.

[0105] This disclosure embodiment can fuse different features of the same object. Specifically, a weight coefficient corresponding to each image feature can be determined based on the image features of different images of the same object. Feature fusion of image features is then performed using this weight coefficient. Since different weight coefficients can be determined for each image feature, the technical solution of this application can improve the accuracy of feature fusion.

[0106] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0107] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0108] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0109] Figure 1 A flowchart illustrating an image processing method according to an embodiment of the present disclosure is shown;

[0110] Figure 2 A flowchart illustrating a method for determining weight coefficients in an image processing method according to an embodiment of the present disclosure is shown.

[0111] Figure 3 This diagram shows a flowchart of step S20 in an image processing method according to an embodiment of the present disclosure;

[0112] Figure 4 This diagram illustrates a flowchart of performing a first optimization process in an image processing method according to an embodiment of the present disclosure;

[0113] Figure 5 This diagram shows a flowchart of step S232 in an image processing method according to an embodiment of the present disclosure;

[0114] Figure 6 This diagram shows a flowchart of step S20 in an image processing method according to an embodiment of the present disclosure;

[0115] Figure 7 This diagram shows a flowchart of step S203 in an image processing method according to an embodiment of the present disclosure;

[0116] Figure 8 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown;

[0117] Figure 9 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown;

[0118] Figure 10 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0119] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0120] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0121] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0122] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0123] This disclosure provides an image processing method that can perform feature fusion processing of multiple images. This method can be applied to any electronic device or server. For example, the electronic device may include user equipment (UE), mobile device, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The server may include a local server or a cloud server. In some possible implementations, the image generation method can be implemented by a processor calling computer-readable instructions stored in memory. The above is merely an exemplary description of the device and is not intended to limit the specific scope of this disclosure. In other embodiments, it can also be implemented by other devices capable of performing image processing.

[0124] Figure 1 A flowchart illustrating an image processing method according to an embodiment of the present disclosure is shown. The image processing method includes:

[0125] S10: Obtain image features from multiple images of the same object;

[0126] In this embodiment, feature fusion processing can be performed on the features of different images of the same object. The object type can be any type, such as a person, animal, plant, vehicle, cartoon character, etc., and this disclosure does not specifically limit this. The different images of the same object can be different images taken in the same scene or images taken in different scenes. Furthermore, this embodiment does not specifically limit the time for acquiring the images; the acquisition time for each image can be the same or different.

[0127] This disclosure embodiment can first acquire multiple images of the same object. Acquiring multiple images can include: capturing the multiple images using a camera device, receiving the multiple images transmitted through communication with other devices, or reading multiple images stored locally or at a specific network address. The above is merely illustrative; in other embodiments, multiple images of the same object can also be obtained through other methods.

[0128] After acquiring an image, image features can be extracted. For example, in some possible implementations, image features can be extracted using feature extraction algorithms, such as face feature extraction algorithms, edge feature extraction algorithms, etc., or other feature extraction algorithms can be used to extract relevant features of the object. Alternatively, embodiments of this disclosure can also extract corresponding image features using a neural network with corresponding functions. These image features can reflect the feature information of the corresponding image or the feature information of objects in the image, such as the grayscale values ​​of each pixel.

[0129] In this embodiment of the disclosure, when the object included in the image is a person, the acquired image features can be the facial features of that object. For example, facial feature extraction algorithms can be used to process each image to extract facial features. Alternatively, each image can be input into a neural network capable of acquiring facial features from the image, and the facial features of each image can be obtained through the neural network. This neural network is a trained neural network capable of acquiring image features from images and then performing object recognition in the image. The result of processing the last convolutional layer of the neural network (the features obtained before classification and recognition) can be used as the image features in this embodiment of the disclosure. The neural network can be a convolutional neural network. Alternatively, for other types of objects, corresponding image features can also be obtained through appropriate feature extraction algorithms or neural networks; this embodiment of the disclosure does not specifically limit this.

[0130] In this embodiment of the disclosure, image features can be in the form of feature vectors. For example, the image features of the i-th image (such as facial features) can be represented as: X i =[x i1 ,x i2 ,x i3 ,...,x iD ], where D represents the dimension of the image features, i is an integer between 1 and N, and N represents the number of images.

[0131] S20: Determine the weight coefficients that correspond one-to-one with each of the image features based on the image features of each image;

[0132] This disclosure allows for the determination of weight coefficients for each image feature based on feature parameters within the image features of each image. These weight coefficients can be values ​​between [0,1] or other values, and this disclosure does not impose specific limitations on them. By configuring different weight coefficients for each image feature, image features with higher precision can be highlighted, thereby improving the accuracy of the fused features obtained through feature fusion processing.

[0133] S30: Based on the weight coefficients of each of the image features, perform feature fusion processing on the image features of the multiple images to obtain the fused features of the multiple images.

[0134] In this embodiment of the disclosure, the method of performing feature fusion processing may include: obtaining the fused features by summing the products of each image feature and its corresponding weight coefficient. For example, the fused features of each image feature can be obtained by the following formula:

[0135]

[0136] Where G represents the generated fusion feature, i is an integer value between 1 and N, N represents the number of images, and b i X represents the image feature of the i-th image. i The weighting coefficients.

[0137] In other words, the embodiments of this disclosure can perform multiplication processing on image features and corresponding weight coefficients, and then sum the multiplication results obtained from each multiplication processing to obtain the fused features of the embodiments of this disclosure.

[0138] Through the embodiments of this disclosure, the weight coefficient corresponding to each image feature can be determined according to the feature parameters in the image features, and the fusion features of each image can be obtained according to the weight coefficient, instead of simply taking the average value of each image feature to obtain the fusion features. This improves the accuracy of the fusion features and also has the advantages of simplicity and convenience.

[0139] The processes of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0140] In this embodiment of the disclosure, after obtaining the image features of different images of the same object, the weight coefficients of each image feature can be determined. In some possible implementations, the weight coefficients can be obtained by feature fitting; in other possible implementations, the weight coefficients can be obtained by median filtering; or in other implementations, the weight coefficients can be obtained by averaging or other processing. This disclosure does not specifically limit the scope of the embodiments.

[0141] Before performing step S20 to obtain each weight coefficient, the method for obtaining each weight coefficient can be determined first, such as feature fitting or median filtering. Figure 2 A flowchart illustrating a method for determining weight coefficients in an image processing method according to an embodiment of the present disclosure is provided. Before determining the weight coefficients corresponding to each image feature based on the image features of each image, the method further includes:

[0142] S41: Obtain the selection information for the acquisition mode of the weight coefficients;

[0143] The selection information refers to the mode selection information for performing the operation of obtaining the weight coefficients. For example, the selection information may be first selection information for obtaining the weight coefficients using a first mode (such as feature fitting), or second selection information for obtaining the weight coefficients using a second mode (such as median filtering). Alternatively, it may include selection information for obtaining the weight coefficients using other modes, which is not specifically limited in this disclosure.

[0144] The method of obtaining the selection information may include receiving input information received by the input component and determining the selection information based on the input information. In this embodiment, the input component may include a switch, keyboard, mouse, audio receiving interface, touchpad, touch screen, communication interface, etc. This disclosure does not specifically limit it; any component capable of receiving selection information can be used as an embodiment of this disclosure.

[0145] S42: Determine the acquisition mode of the weight coefficients based on the selection information;

[0146] Since the selection information includes information about the acquisition mode of the weight information, the corresponding mode information can be obtained based on the received selection information. For example, when the selection information includes first selection information, it can be determined that the first mode (feature fitting method) is used to acquire the weight coefficients; when the selection information includes second selection information, it can be determined that the second mode (median filtering method) is used to acquire the weight coefficients. Correspondingly, when the selection information includes other selection information, the method for acquiring the weight coefficients corresponding to the selection information can be determined accordingly.

[0147] In some possible implementations, the precision, computational load, and computational speed of different weight coefficient acquisition modes may differ. For example, the precision of the first mode may be higher than that of the second mode, and the computational speed of the first mode may be lower than that of the second mode, but this is not a specific limitation of this disclosure. Therefore, in the embodiments of this disclosure, users can select an appropriate mode to acquire weight parameters according to different needs.

[0148] S43: Based on the determined acquisition mode of the weight coefficients, perform the step of determining the weight coefficients corresponding to each image feature according to the image features of each image; wherein, the acquisition mode of the weight coefficients includes obtaining the weight coefficients by feature fitting and obtaining the weight coefficients by median filtering.

[0149] After determining the acquisition mode of the weight coefficients based on the selection information, the weight information acquisition operation can be performed according to the determined mode.

[0150] In this embodiment of the disclosure, the selection of the weight coefficient acquisition mode can be realized through the above method. Under different needs, different modes can be used to acquire the weight coefficient, which has better applicability.

[0151] The method for obtaining weight coefficients in the embodiments of this disclosure will be described in detail below. Figure 3 This diagram illustrates a flowchart of step S20 in an image processing method according to an embodiment of the present disclosure, wherein determining the weight coefficients corresponding to each image feature based on the image features of each image (step S20) may include:

[0152] S21: Form an image feature matrix based on the image features of each image;

[0153] In this embodiment of the disclosure, the image features of each image can be represented as feature vectors. For example, the image feature of the i-th image can be represented as X. i =[x i1 ,x i2 ,x i3 ,...,x iD ], where D represents the dimension of the image features, i is an integer between 1 and N, and N represents the number of images. Furthermore, in this embodiment of the disclosure, the dimension of the image features of each image is the same, which is D.

[0154] The image feature matrix X, formed based on the image features of each image, can be represented as:

[0155]

[0156] Based on the above, an image feature matrix composed of each image feature can be obtained. In the above method, the elements of each row in the image feature matrix can be considered as image features of one image, and the image features corresponding to each row are image features of different images. In other embodiments, the elements of each column in the image feature matrix can also be considered as image features of one image, and the image features corresponding to each column are image features of different images. This disclosure does not specifically limit the arrangement of the image feature matrix.

[0157] S22: Perform feature fitting processing on the image feature matrix to obtain the first weight matrix;

[0158] After obtaining the image feature matrix corresponding to each image feature, feature fitting processing of the image feature matrix can be performed. In this embodiment, regularized least-squares linear regression can be used to perform this feature fitting processing. For example, a preset objective function can be set, which is a function related to the weight coefficients. When the preset objective function reaches its minimum value, a first weight matrix corresponding to each weight coefficient is determined. The dimension of the first weight matrix is ​​the same as the number of image features, and the final weight coefficients can be determined based on each element in the first weight matrix.

[0159] In some possible implementations, the expression for the preset objective function can be:

[0160]

[0161] Where X represents the image feature matrix, and b = [b1, b2, ..., b N ] T Let Y represent the first weight matrix to be estimated, which can be represented by Y, which is the same as X. T Let X be the transpose matrix, and λ be the regularization parameter. This represents the L2norm (standard) regularization term for the parameter.

[0162] In some possible implementations, if the image features are row vectors, the generated first weight matrix is ​​a column vector; conversely, if the image features are column vectors, the generated first weight matrix is ​​a row vector. Furthermore, the dimension of the first weight matrix is ​​the same as the number of image features or images.

[0163] This embodiment of the disclosure can determine the value of the first weight matrix b when the above objective function is minimized. The final first weight matrix can then be obtained, and its expression can be:

[0164] b = (X TX+λI) -1 X T Y.

[0165] Through the above embodiments, the first weight matrix obtained by feature fitting processing can be obtained. In other embodiments of this disclosure, feature fitting processing of the image feature matrix can also be performed by other feature fitting methods to obtain the corresponding first weight matrix, or different preset objective functions can be set to perform feature fitting processing. This disclosure does not specifically limit this.

[0166] S23: Determine the weight coefficients corresponding to each image feature based on the first weight matrix.

[0167] After obtaining the first weight matrix, the weight coefficients corresponding to the image features can be determined based on the obtained first weight matrix.

[0168] In some possible implementations, each element in the first weight matrix can be directly used as a weight coefficient; that is, each first weight coefficient in the first weight matrix can be used as an image feature weight coefficient. The resulting first weight matrix is ​​b = [b1, b2, ..., b...]. N ] T In the case of the i-th image, the image feature X i The weighting coefficient can be b. i .

[0169] In other embodiments of this disclosure, to further improve the accuracy of the weight coefficients, an optimization process can be performed on the first weight matrix to obtain an optimized first weight matrix, and the elements in the optimized first weight matrix can be used as the weight coefficients for each image feature. That is, a first optimization process can be performed on the first weight matrix, and each first weight coefficient included in the optimized first weight matrix can be determined as the weight coefficient corresponding to each image feature. This first optimization process can detect outliers in the first weight matrix, and corresponding optimization processes can be performed on these outliers to improve the accuracy of the obtained weight matrix.

[0170] Figure 4 The flowchart illustrates the execution of a first optimization process in an image processing method according to an embodiment of the present disclosure. The first optimization process is performed on the first weight matrix, and each first weight coefficient included in the optimized first weight matrix is ​​determined to be the weight coefficient corresponding to each image feature. This may include:

[0171] S231: Based on the first weight coefficients of each image feature included in the first weight matrix, determine the fitted image features of each image, wherein the fitted image features are the product of the image features and the corresponding first weight coefficients;

[0172] In this embodiment of the disclosure, fitted image features for each image feature can first be obtained based on a determined first weight matrix. Specifically, the first weight coefficients of each image feature included in the first weight matrix can be multiplied with the corresponding image feature to obtain the fitted image feature for that image feature. For example, the image feature X of the i-th image in the first weight matrix can be... i First weighting coefficient b i With the image feature X i Multiply to obtain the fitted image features b i X i .

[0173] S232: Using the first error between the image features of each image and the fitted image features, perform the first optimization process on the first weight matrix to obtain the first optimized weight matrix;

[0174] After obtaining the fitted image features, a first error between the image features and their corresponding fitted image features can be obtained. In this embodiment, the first error between the image features and the fitted image features can be obtained according to the following formula:

[0175]

[0176] Among them, e i Let X represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer from 1 to N, N is the number of image features, j is an integer from 1 to D, and D represents the dimension of each image feature. i b represents the image features of the i-th image. i X i This represents the fitted image feature corresponding to the i-th image feature.

[0177] In other embodiments of this disclosure, the first error between the image features and the fitted image features can also be determined in other ways. For example, the average value of the differences between each element of the fitted image features and the image features can be directly used as the first error. This disclosure does not specifically limit the method of determining the first error.

[0178] After obtaining the first error, the first optimization process of the first weight matrix can be performed using this first error to obtain the first optimized weight matrix. The elements in this first optimized weight matrix can also represent the first optimized weight coefficients corresponding to each image feature.

[0179] S233: Determine whether the difference between the first weight matrix and the first optimized weight matrix satisfies the first condition. If yes, proceed to step S234; otherwise, proceed to step S235.

[0180] After obtaining the first optimization result (first optimized weight matrix) of the first weight matrix based on the first error, it can be determined whether the difference between the first optimized weight matrix and the first weight matrix satisfies the first condition. If the difference satisfies the first condition, it means that the first optimized weight matrix does not need to be further optimized, and the first optimized weight matrix can be determined as the final optimized weight matrix obtained from the first optimization process. If the difference between the first optimized weight matrix and the first weight matrix does not satisfy the first condition, then the first optimized weight matrix needs to be further optimized.

[0181] In this embodiment of the present disclosure, the first condition can be that the absolute value of the difference between the first optimized weight matrix and the first weight matrix is ​​less than a first threshold. The first threshold is a preset threshold, which can be a value less than 1. In this embodiment of the present disclosure, the value of the first threshold can be set according to the requirements, and the present disclosure does not make a specific limitation on it. For example, it can be 0.01.

[0182] Based on the above embodiments, it is possible to determine whether the difference between the first optimized weight matrix and the first weight matrix satisfies the first condition, and then further execute the corresponding subsequent steps.

[0183] S234: Determine the first optimized weight matrix as the optimized first weight matrix;

[0184] As described in the above embodiments, if it is determined that the difference between the first optimized weight matrix and the first weight matrix satisfies the first condition, it means that the first optimized weight matrix does not need to undergo further optimization processing. At this time, the first optimized weight matrix can be directly determined as the final optimized weight matrix obtained from the first optimization processing.

[0185] S235: Obtain new fitted image features using the first optimized weight matrix, and repeat the first optimization process based on the new fitted image features until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix satisfies the first condition. Then, determine the k-th optimized weight matrix as the optimized first weight matrix, where k is a positive integer greater than 1.

[0186] In some possible implementations, based on the first error between the image features and the fitted image features, the difference between the first optimized weight matrix and the first weight matrix obtained by the first optimization processing of the image features may not satisfy the first condition, for example, if the difference is greater than the first threshold. In this case, the weight coefficients in the first optimized weight matrix can be used to obtain the fitted image features of each image feature. Then, the first error between the image features and the fitted image features is used to further perform the second first optimization processing to obtain the second optimized weight matrix.

[0187] If the difference between the second optimized weight matrix and the first optimized weight matrix satisfies the first condition, the second optimized weight matrix can be determined as the final optimization result, i.e., the optimized weight matrix. If the difference between the second optimized weight matrix and the first optimized weight matrix still does not satisfy the first condition, the weight coefficients in the second optimized weight matrix can be used to obtain the fitted image features of each image feature, and the first error between the image feature and the fitted image feature can be used to further perform the third first optimization process to obtain the third optimized weight matrix. This process continues until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix satisfies the first condition. At this time, the k-th optimized weight matrix can be determined as the optimized first weight matrix, where k is a positive integer greater than 1.

[0188] The above embodiments enable the process of performing a first optimization process based on the first error between image features and fitted image features, and obtaining an optimized first weight matrix. In this embodiment, the expression for the iterative function of the first optimization process can be:

[0189] b (t) =(X T W (t-1) X+λI) -1 X T W (t-1) Y;

[0190] Where t represents the number of iterations (i.e., the number of times the first optimization process is performed), b (t) Let W represent the first optimized weight matrix obtained from the t-th first optimization process, X represent the image feature matrix, Y represent the observation matrix (which is the same as X), and W represent the first optimized weight matrix obtained from the first optimization process. (t-1) w represents the second weight coefficient obtained in the (t-1)th iteration. i Let I be a diagonal matrix, and λ represent the regularization parameter. From the above embodiments, it can be seen that the embodiments of this disclosure can adjust the second weight coefficient w each time the first optimization process is performed. i The weight matrix is ​​optimized.

[0191] This disclosure describes the first optimization process in conjunction with the first optimization process of the first weight matrix. Figure 5 The flowchart illustrates step S232 of an image processing method according to an embodiment of the present disclosure. The step of performing a first optimization process on the first weight matrix using a first error between the image features of each image and the fitted image features includes:

[0192] S2321: The first error between the image features and the fitted image features is obtained by summing the squares of the differences between the corresponding elements in each image feature and the fitted image features;

[0193] As described in the above embodiments, after obtaining the image features and the corresponding fitted image features, a first error between each image feature and the corresponding fitted image feature can be determined.

[0194]

[0195] S2322: Obtain the second weighting coefficients for each image feature based on each of the first errors;

[0196] After determining the first error between each image feature and its corresponding fitted image feature, a second weighting coefficient for the image feature can be determined based on the value of the first error, for use in performing the first optimization process. The second weighting coefficient for the corresponding image feature can be determined using a first method, the expression of which is:

[0197]

[0198] Among them, w i e is the second weighting coefficient for the i-th image. i Let represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer between 1 and N, N is the number of image features, and k = 1.345σ, where σ is the error e. i Standard deviation. In this embodiment, k can represent an error threshold, which can be 1.348 = 5 times the standard deviation of the first error between all image features and the fitted image features. In other embodiments, the value of k can be other values, such as 0.6, etc., which are not intended to limit the scope of this disclosure.

[0199] After obtaining the first error between each image feature and the fitted image feature, this first error can be compared with an error threshold k. If the first error is less than k, the second weight coefficient corresponding to the image feature can be determined as a first value, such as 1. If the first error is greater than or equal to k, the second weight coefficient of the image feature can be determined based on the first error. In this case, the second weight coefficient can be a second value, which is the ratio of the absolute value of k to the second error.

[0200] S2323: Perform the first optimization process on the first weight matrix based on the second weight coefficients of each image to obtain the first optimized weight matrix.

[0201] After obtaining the second weight coefficients of the image features, the first optimization process of the first weight matrix can be performed using these second weight coefficients. This optimization can be achieved using the iterative function b. (t) =(X T W (t-1) X+λI) -1X T W (t-1) Y obtains the first optimized weight matrix.

[0202] In this embodiment of the disclosure, if the difference between the first optimized weight matrix and the first weight matrix does not satisfy the first condition, after obtaining new fitted image features using the weight coefficients in the first weight matrix, the second weight coefficients of each image feature can be re-determined based on the first error between the image feature and the new fitted image feature. Then, the above function iteration is performed based on the new second weight coefficients to obtain the second optimized weight matrix. Similarly, the k-th optimized weight matrix corresponding to the k-th first optimization process can be obtained.

[0203] Therefore, the difference between the k-th optimized weight matrix obtained in the k-th first optimization process and the (k-1)-th optimized weight matrix obtained in the (k-1)-th first optimization process satisfies the first condition b. (t-1) -b (t) |<ε where ε is the first threshold, then the k-th optimized weight matrix b can be... (t) This serves as the first weight matrix after optimization.

[0204] Based on the above embodiments, the process of obtaining the weight coefficients of image features through feature fitting can be completed. The weight coefficients obtained by this method have high accuracy and are also more robust to outliers in the weight coefficients.

[0205] As described above, this disclosure also provides a method for determining the weight coefficients of each image feature using median filtering. This method has lower computational cost compared to feature fitting.

[0206] Figure 6 The flowchart illustrates step S20 of an image processing method according to an embodiment of the present disclosure, wherein determining the weight coefficients corresponding to each image feature based on the image features of each image (step S20) may further include:

[0207] S201: Form an image feature matrix based on the image features of each image;

[0208] Similar to step S21, in this embodiment of the disclosure, an image feature matrix can be formed based on the image features of each image. The image features of each image can be represented as feature vectors. For example, the image features of the i-th image can be represented as X. i =[x i1 ,x i2 ,x i3 ,...,x iD], where D represents the dimension of the image features, i is an integer between 1 and N, and N represents the number of images. Furthermore, in this embodiment of the disclosure, the dimension of the image features of each image is the same, which is D.

[0209] The image feature matrix X, formed based on the image features of each image, can be represented as:

[0210]

[0211] Based on the above, an image feature matrix composed of each image feature can be obtained. In the above method, the elements of each row in the image feature matrix can be considered as image features of one image, and the image features corresponding to each row are image features of different images. In other embodiments, the elements of each column in the image feature matrix can also be considered as image features of one image, and the image features corresponding to each column are image features of different images. This disclosure does not specifically limit the arrangement of the image feature matrix.

[0212] S202: Perform median filtering on the image feature matrix to obtain the median feature matrix;

[0213] In this embodiment of the disclosure, after obtaining the image feature matrix, median filtering can be performed on the obtained image feature matrix to obtain the median feature matrix corresponding to the image feature matrix. The elements in the median feature matrix are the median values ​​of the image features corresponding to the respective elements in the image feature matrix.

[0214] In this embodiment of the disclosure, the median value of each image feature in the image feature matrix for the same position can be determined; the median feature matrix is ​​obtained based on the median value of the elements at each position.

[0215] For example, the image feature matrix of this embodiment can be represented as Correspondingly, the median of the image features at each corresponding location can be obtained. Here, "location" refers to the position corresponding to the ordinal number of the feature within each image feature; for example, the first element in each image feature could be (x... 11 x 21 , ..., x N1 Alternatively, the j-th element at position j can be (x 1j x 2j , ..., x Nj The elements at the same position can be determined through the above process. The dimension of the median feature matrix obtained in this embodiment can be the same as the dimension of the image features, and the median feature matrix can be represented as M = [m1, m2, ..., m...]. D ], where any j-th element can be m j =median([m 1j ,m2j ,...,m Nj ]), where j is an integer value between 1 and D. The median function is the median function, which yields [m 1j ,m 2j ,...,m Nj The value in the middle of the eigenvalues ​​in the equation is [m]. We can first consider the values ​​in the middle of the eigenvalues ​​in the equation. 1j ,m 2j ,...,m Nj Sort the data from largest to smallest. When N is odd, the median is the image feature value (element value) at the middle position ((N+1) / 2). When N is even, the median is the average of the two middle element values.

[0216] Based on the above, the median feature matrix corresponding to each image feature in the image feature matrix can be obtained.

[0217] S203: Determine the weight coefficients corresponding to each image feature based on the median feature matrix.

[0218] After obtaining the median feature matrix corresponding to the image features, the weight coefficients of the image features can be obtained using the median.

[0219] In some possible implementations, a second error between each image feature and the median feature matrix can be used, and the weight coefficient of each image feature can be determined based on the second error.

[0220] Figure 7 A flowchart of step S203 in an image processing method according to an embodiment of the present disclosure is shown. The step of determining the weight coefficients corresponding to each image feature based on the median feature matrix includes:

[0221] S2031: Obtain the second error between each image feature and the median feature matrix;

[0222] In this embodiment of the disclosure, the sum of the absolute values ​​of the differences between corresponding elements in the image features and the median feature matrix can be used as the second error between the image features and the median feature matrix. The expression for the second error can be:

[0223]

[0224] Among them, e h The image features X of the h-th image h The second error between the median eigenvalue matrix and the median eigenvalue matrix, where M represents the median eigenvalue matrix and X... h This represents the image feature of the h-th image, where h is an integer value between 1 and N.

[0225] Through the above embodiments, a second error between each image feature and the median feature matrix can be obtained, and then the weighting coefficients can be determined through the second error.

[0226] S2032: Determine whether the second error meets the second condition. If yes, proceed to step S2033; otherwise, proceed to step S2034.

[0227] In this embodiment of the present disclosure, the second condition can be that the second error is greater than a second threshold. This second threshold can be a preset value, or it can be determined by the second error between each image feature and the median feature matrix; the present disclosure does not specifically limit this. In some possible implementations, the expression for the second condition can be:

[0228] e h >K·MADN;

[0229] MADN = median([e1,e2,...e...) N ]) / 0.675;

[0230] Among them, e h Let K be the second error between the image features of the h-th image and the median feature matrix, where h is an integer value from 1 to N, N represents the number of images, K is the judgment threshold, which can be a set value, such as 0.8, but is not a limitation of this embodiment, and median represents the median filtering function. That is, in this embodiment, the second threshold can be the product of the ratio of the mean of the second errors corresponding to each image feature to 0.675 and the judgment threshold K, and the judgment threshold can be a positive number less than 1.

[0231] By setting a second condition or a second threshold, it can be determined whether the second error between the image features and the median feature matrix meets the second condition, and subsequent operations can be performed based on the determination result.

[0232] S2033: Configure the weight coefficient of this image feature as the first weight;

[0233] In this embodiment of the present disclosure, when the second error between the image feature and the median feature matrix satisfies a second condition, such as the second error being greater than a second threshold, it indicates that the image feature may be abnormal. In this case, the first weight can be determined as the weight coefficient of the image feature. The first weight in this embodiment can be a preset weight coefficient, for example, it can be 0. Alternatively, in other embodiments, the first weight can be set to other values ​​to reduce the impact of potentially abnormal image features on the fused features.

[0234] S2034: Determine the weight coefficients of the image features using the second method.

[0235] In this embodiment of the disclosure, when the second error between the image feature and the median feature matrix does not satisfy the second condition (e.g., the second error is less than or equal to the second threshold), it indicates that the image feature is relatively accurate. Therefore, the weight coefficient of the image feature can be determined based on the second error using the second method. The expression for the second method can be:

[0236]

[0237] θ h =1 / e h ;

[0238] Among them, b h Let e ​​be the weight coefficient of the h-th image determined by the second method. h The second error between the image features of the h-th image and the median feature matrix is ​​denoted as h, where h is an integer value from 1 to N, and N represents the number of images.

[0239] When the second error corresponding to the image feature is less than or equal to the second threshold, the weight coefficient b of the image feature can be obtained through the second method described above. h .

[0240] Based on the embodiments of this disclosure, the weight coefficients of each image feature can be obtained by median filtering. The median filtering method for determining the weight coefficients can further reduce computational overhead, effectively reduce the complexity of computation and processing, and improve the accuracy of the obtained fused features.

[0241] After obtaining the weight coefficients of each image feature, feature fusion processing can be performed. For example, the fused feature can be obtained by summing the products of each image feature and its corresponding weight coefficient.

[0242] In some possible implementations of this disclosure, after obtaining the fusion features, the embodiments of this disclosure can also utilize the fusion features to perform target object recognition operations in the image. For example, the fusion features can be compared with images of various objects stored in a database. If a first image exists with a similarity greater than a similarity threshold, the target object can be identified as the object corresponding to that first image, thereby completing the identity recognition and target recognition operations. In other embodiments of this disclosure, recognition operations for other types of objects can also be performed, and this disclosure does not specifically limit them.

[0243] To more clearly illustrate the process of the embodiments of this disclosure, a human face image will be used as an example for explanation below.

[0244] This embodiment of the disclosure can first acquire different facial images of object A, for example, N facial images, where N is an integer greater than 1. After acquiring these N facial images, facial features can be extracted from these N facial images using a neural network capable of extracting facial features, forming facial features (image features) X for each image. i =[x i1 ,x i2 ,x i3 ,...,x iD ].

[0245] After obtaining the facial features of each face image, the weight coefficient corresponding to each facial feature can be determined. In this embodiment, the weight coefficient can be obtained using feature fitting or median filtering, depending on the received selection information. When using feature fitting, the facial feature matrix corresponding to each facial feature can be obtained first. The first weight matrix is ​​obtained by feature fitting of the image features. This first weight matrix can be represented as b = (X T X+λI) -1 X T Y, and then the first optimization process can be performed on the first weight matrix, where the iterative function of the first optimization process is expressed as b. (t) =(X T W (t-1) X+λI) -1 X T W (t-1) Y, obtain the optimized first weight matrix, and determine the weight coefficient of each face feature based on the parameters in the optimized first weight matrix.

[0246] When obtaining weight coefficients using median filtering, the image feature matrix can also be obtained. Then, by obtaining the median of the elements at the same position for each image feature in the image feature matrix, the median feature matrix M = [m1, m2, ..., m] can be determined. D Then, the weight coefficients of the image features are determined based on the second error between each image feature and the median feature matrix.

[0247] After obtaining the weight coefficients of each image feature, the fused feature can be obtained by summing the products of the weight coefficients and the image features. This fused feature can then be used to perform operations such as object detection and object recognition. The above description is merely an illustrative example of the feature fusion process in this embodiment and is not intended to limit the scope of this disclosure.

[0248] In summary, the embodiments of this disclosure can fuse different features of the same object. Specifically, a weight coefficient corresponding to each image feature can be determined based on the image features of different images of the same object, and feature fusion of image features can be performed through the weight coefficient. This method can improve the accuracy of feature fusion.

[0249] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0250] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0251] In addition, this disclosure also provides an image processing apparatus, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the image processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section and will not be repeated here.

[0252] Figure 8 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown, such as Figure 8 The image processing apparatus shown in this embodiment of the disclosure may include:

[0253] The acquisition module 10 is used to acquire image features of multiple images of the same object respectively;

[0254] The determining module 20 is used to determine the weight coefficients corresponding one-to-one with each of the image features based on the image features of each image;

[0255] The fusion module 30 is used to perform feature fusion processing on the image features of the plurality of images based on the weight coefficients of each of the image features, so as to obtain the fused features of the plurality of images.

[0256] In some possible implementations, the determining module includes:

[0257] The first establishing unit is used to form an image feature matrix based on the image features of each image;

[0258] A fitting unit is used to perform feature fitting processing on the image feature matrix to obtain a first weight matrix;

[0259] The first determining unit is used to determine the weight coefficients corresponding to each image feature based on the first weight matrix.

[0260] In some possible implementations, the fitting unit is further configured to perform feature fitting processing on the image feature matrix using a regularized linear least squares estimation algorithm, and obtain the first weight matrix when the preset objective function is minimized.

[0261] In some possible implementations, the determining module further includes an optimization unit for performing a first optimization process on the first weight matrix;

[0262] The first determining unit is further configured to determine each first weight coefficient included in the first weight matrix as the weight coefficient corresponding to each image feature; or to determine each first weight coefficient included in the optimized first weight matrix as the weight coefficient corresponding to each image feature.

[0263] In some possible implementations, the optimization unit is further configured to determine the fitted image features of each image based on the first weight coefficients of each image feature included in the first weight matrix.

[0264] Using the first error between the image features of each image and the fitted image features, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix;

[0265] In response to the first condition being met by the difference between the first weight matrix and the first optimized weight matrix, the first optimized weight matrix is ​​determined as the optimized first weight matrix.

[0266] In response to the first condition not being met by the difference between the first weight matrix and the first optimized weight matrix, new fitting image features are obtained using the first optimized weight matrix. The first optimization process is repeated based on the new fitting image features until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix meets the first condition. The k-th optimized weight matrix is ​​then determined as the optimized first weight matrix, where k is a positive integer greater than 1.

[0267] Wherein, the fitted image feature is the product of the image feature and the corresponding first weight coefficient.

[0268] In some possible implementations, the optimization unit is further configured to obtain a first error between the image features and the fitted image features based on the sum of squares of the differences between corresponding elements in each image feature and the fitted image features;

[0269] The second weighting coefficients for each image feature are obtained based on each of the first errors;

[0270] Based on the second weight coefficients of each image, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix corresponding to the first weight matrix.

[0271] In some possible implementations, the optimization unit is further configured to obtain second weight coefficients for each image feature based on each of the first errors using a first method, wherein the expression for the first method is:

[0272]

[0273] Among them, w i e is the second weighting coefficient for the i-th image. i Let represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer between 1 and N, N is the number of image features, and k = 1.345σ, where σ is the error e. i The standard deviation.

[0274] In some possible implementations, the determining module further includes:

[0275] The second establishing unit is used to form an image feature matrix based on the image features of each image;

[0276] A filtering unit is used to perform median filtering on the image feature matrix to obtain a median feature matrix;

[0277] The second determining unit is used to determine the weight coefficients corresponding to each image feature based on the median feature matrix.

[0278] In some possible implementations, the filtering unit is also used to determine the median value of each image feature in the image feature matrix for the same position;

[0279] The median feature matrix is ​​obtained based on the median value of the elements at each position.

[0280] In some possible implementations, the second determining unit is further configured to obtain a second error between each image feature and the median feature matrix;

[0281] In response to the second error between the image feature and the median feature matrix satisfying the second condition, the weight coefficient of the image feature is configured as the first weight. In response to the second error between the image feature and the median feature matrix not satisfying the second condition, the weight coefficient of the image feature is determined using the second method.

[0282] In some possible implementations, the expression for the second method is:

[0283]

[0284] θ h =1 / e h ;

[0285] Among them, b h Let e ​​be the weight coefficient of the h-th image determined by the second method. h The second error between the image features of the h-th image and the median feature matrix is ​​denoted as h, where h is an integer value from 1 to N, and N represents the number of images.

[0286] In some possible implementations, the second condition is:

[0287] e h >K·MADN;

[0288] MADN = median([e1,e2,...e...) N ]) / 0.675;

[0289] Among them, e h Let be the second error between the image features of the h-th image and the median feature matrix, where h is an integer value from 1 to N, N represents the number of images, K is the judgment threshold, and median represents the median filtering function.

[0290] In some possible implementations, the fusion module is further configured to obtain the fused features by summing the products of each image feature and its corresponding weight coefficient.

[0291] In some possible implementations, the device further includes an identification module for performing the identification operation of the same object using the fusion features.

[0292] In some possible implementations, the apparatus further includes a mode determination module for selecting information on the acquisition mode of the weight coefficients, and determining the acquisition mode of the weight coefficients based on the selection information. The acquisition mode of the weight coefficients includes acquiring the weight coefficients by feature fitting and acquiring the weight coefficients by median filtering.

[0293] The determining module is further configured to, based on the determined acquisition mode of the weight coefficients, perform the step of determining the weight coefficients corresponding to each image feature according to the image features of each image.

[0294] In some embodiments, the apparatus provided in this disclosure may have functions or include modules that can be used to perform the methods described in the above method embodiments. Specific implementations can be referred to the descriptions in the above method embodiments, and for brevity, will not be repeated here.

[0295] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0296] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured as described above.

[0297] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0298] Figure 9 This diagram illustrates a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or other terminal.

[0299] Reference Figure 9 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0300] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0301] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0302] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0303] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0304] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0305] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0306] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0307] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0308] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0309] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of an electronic device 800 to perform the above-described method.

[0310] Figure 10 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 10 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0311] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0312] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0313] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0314] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0315] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0316] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0317] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0318] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0319] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0320] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0321] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized in that, include: Obtain image features from multiple images of the same object; Based on the image features of each image, determine the weight coefficients that correspond one-to-one with each of the image features; Based on the weight coefficients of each of the image features, feature fusion processing is performed on the image features of the multiple images to obtain the fused features of the multiple images; The step of determining the weight coefficients corresponding one-to-one with each image feature based on the image features of each image includes: Based on the image features of each image, an image feature matrix is ​​formed; Perform feature fitting processing on the image feature matrix to obtain the first weight matrix; Perform a first optimization process on the first weight matrix, and determine each first weight coefficient included in the optimized first weight matrix as the weight coefficient corresponding to each image feature, so as to detect outliers in the first weight matrix; Before determining the weight coefficient corresponding to each image feature based on the image features of each image, the method further includes: obtaining selection information for the acquisition mode of the weight coefficient; determining the acquisition mode of the weight coefficient based on the selection information; and performing the step of determining the weight coefficient corresponding to each image feature based on the determined acquisition mode of the weight coefficient.

2. The method according to claim 1, characterized in that, The step of performing feature fitting processing on the image feature matrix to obtain a first weight matrix includes: The image feature matrix is ​​subjected to feature fitting processing using a regularized linear least squares estimation algorithm, and the first weight matrix is ​​obtained when the preset objective function is minimized.

3. The method according to claim 1 or 2, characterized in that, The first optimization process performed on the first weight matrix includes: Based on the first weight coefficients of each image feature included in the first weight matrix, the fitted image features of each image are determined, wherein the fitted image features are the product of the image features and the corresponding first weight coefficients. Using the first error between the image features of each image and the fitted image features, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix; In response to the first condition being met by the difference between the first weight matrix and the first optimized weight matrix, the first optimized weight matrix is ​​determined as the optimized first weight matrix. If the difference between the first weight matrix and the first optimized weight matrix does not satisfy the first condition, new fitted image features are obtained using the first optimized weight matrix. The first optimization process is repeated based on the new fitted image features until the difference between the obtained k-th optimized weight matrix and the (k-1)-th optimized weight matrix satisfies the first condition. The k-th optimized weight matrix is ​​then determined as the optimized first weight matrix, where k is a positive integer greater than 1.

4. The method according to claim 3, characterized in that, The first optimization process of the first weight matrix, which utilizes the first error between the image features of each image and the fitted image features, includes: The first error between the image features and the fitted image features is obtained by summing the squares of the differences between the corresponding elements in each image feature and the fitted image features. The second weighting coefficients for each image feature are obtained based on each of the first errors; Based on the second weight coefficients of each image, the first optimization process of the first weight matrix is ​​performed to obtain the first optimized weight matrix corresponding to the first weight matrix.

5. The method according to claim 4, characterized in that, The second weighting coefficients for each image feature obtained based on each of the first errors include: Using a first method, second weighting coefficients for each image feature are obtained based on each of the first errors, wherein the expression for the first method is: Among them, w i e is the second weighting coefficient for the i-th image. i Let represent the first error between the i-th image feature and its corresponding fitted image feature, where i is an integer between 1 and N, N is the number of image features, and k = 1.345σ, where σ is the error e. i The standard deviation.

6. The method according to any one of claims 1-5, characterized in that, The step of performing feature fusion processing on the image features of the multiple images based on the weight coefficients of each of the image features to obtain the fused features of the multiple images includes: The fused features are obtained by summing the products of each image feature and its corresponding weight coefficient.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: The same object is identified using the fusion features.

8. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire image features from multiple images of the same object. The determining module is used to determine the weight coefficients corresponding one-to-one with each of the image features based on the image features of each image; The fusion module is used to perform feature fusion processing on the image features of the multiple images based on the weight coefficients of each image feature, to obtain the fused features of the multiple images. The determining module includes: The first establishing unit is used to form an image feature matrix based on the image features of each image; A fitting unit is used to perform feature fitting processing on the image feature matrix to obtain a first weight matrix; An optimization unit is configured to perform a first optimization process on the first weight matrix and determine each first weight coefficient included in the optimized first weight matrix as the weight coefficient corresponding to each image feature, so as to detect outliers in the first weight matrix. The device further includes a mode determination module, which is used to select information on the acquisition mode of the weight coefficients and determine the acquisition mode of the weight coefficients based on the selection information. The determining module is further configured to, based on the determined acquisition mode of the weight coefficients, perform the step of determining the weight coefficients corresponding to each image feature according to the image features of each image.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images