Retrieval method and device, electronic equipment and storage medium

Abstract

The invention provides a retrieval method and device, electronic equipment and a storage medium, and relates to the technical field of data retrieval, and the method comprises the following steps: determining a first attribute value matrix and a first confidence coefficient matrix of a target object; determining a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and a second confidence matrix of the reference object; determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and a second attribute value matrix of the reference object; determining the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix; and determining a retrieval result based on the similarity of each attribute. According to the method, the attribute similarity between the target object and the reference object is determined, and the similarity between the attribute values of the target object and the reference object is comprehensively reflected through the attribute similarity, so that the reference object similar to the target object is retrieved, and retrieval results are richer.

Description

Technical Field

[0001] The present invention relates to the technical field of data retrieval, and in particular to a retrieval method, device, electronic equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application of captured data (pictures or video frames) of smart devices such as smart cameras or servers is becoming more and more extensive, and the retrieval of large amounts of captured data is one of the important applications.

[0003] In the related art, the attribute value of the attribute corresponding to the target in the object to be retrieved is usually determined, and a search is performed in the database based on the attribute value to obtain the object with the same attribute value, and the object with the same attribute value is output as the search result.

[0004] However, in the above-mentioned related technologies, when performing search and comparison, the search is only based on the attribute value of the specific attribute, and the attribute value is a specific value that represents a certain feature of the target. For example, the attribute value of the color attribute of a vehicle is white. In this way, when searching, all objects in the database whose color attribute value is white will be output as search results, and objects whose color attribute value is not white will be excluded, so that the white attribute value has a greater impact on the search results, and it is impossible to comprehensively evaluate the impact of each attribute value on the search results. Summary of the invention

[0005] The present invention provides a retrieval method, device, electronic device and storage medium, which are used to solve the problem that the prior art cannot comprehensively evaluate the influence of each attribute value on the retrieval result.

[0006] The present invention provides a retrieval method, comprising:

[0007] Determine a first attribute value matrix and a first confidence matrix of the target object; each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; and the second elements corresponding to each first element in the first confidence matrix are used to represent the confidence of the corresponding attribute value;

[0008] For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object;

[0009] Determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0010] Determining attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0011] The search result is determined based on the similarity of each attribute.

[0012] According to a retrieval method provided by the present invention, determining a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and the second confidence matrix of the reference object comprises:

[0013] For each of the second elements in the first confidence matrix, a correlation operation is performed between the second element and the third element in the second confidence matrix at the same position as the second element to obtain the confidence correlation matrix.

[0014] According to a retrieval method provided by the present invention, determining the attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object includes:

[0015] For each of the first elements in the first attribute value matrix, determine a difference between the first element and a fourth element in the second attribute value matrix at the same position as the first element, to obtain a difference matrix;

[0016] Performing fixed value processing on each fifth element in the difference matrix to obtain a fixed value result corresponding to each fifth element; the fixed value result is used to indicate whether the attribute of the target object is the same as the attribute of the reference object;

[0017] The attribute distance matrix is ​​determined based on each of the determination results.

[0018] According to a retrieval method provided by the present invention, the determining of retrieval results based on the similarity of each attribute includes:

[0019] Determining a first characteristic value of the target object;

[0020] For each of the reference objects, determining a feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object;

[0021] Determining a mixed similarity based on the feature similarity and the corresponding attribute similarity;

[0022] The search result is determined based on each of the mixed similarities.

[0023] According to a retrieval method provided by the present invention, for each of the reference objects, determining the feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object includes:

[0024] In the case that the number of the reference objects is less than a preset number, deleting the attribute similarities less than a first preset value among all the attribute similarities, and determining the reference objects corresponding to the remaining attribute similarities as the first reference objects;

[0025] For each of the first reference objects, determining a first feature similarity between the target object and the first reference object based on the first feature value and a second feature value of the first reference object;

[0026] The determining of the mixed similarity based on the feature similarity and the corresponding attribute similarity includes:

[0027] The mixed similarity is determined based on the first feature similarity and the corresponding remaining attribute similarities.

[0028] According to a retrieval method provided by the present invention, for each of the reference objects, determining the feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object includes:

[0029] When the number of the reference objects is greater than or equal to the preset number, clustering the second eigenvalues ​​of all the reference objects to obtain at least two clusters;

[0030] For each of the clusters, determining a second feature similarity between the target object and a reference object in the cluster based on the first feature value and a second feature value corresponding to the cluster;

[0031] Deleting all the second feature similarities that are less than a second preset value among the second feature similarities, and determining clusters corresponding to the remaining second feature similarities as target clusters;

[0032] For each second reference object in the target cluster, determining a second feature similarity between the target object and the second reference object based on the first feature value and the second feature value of the second reference object;

[0033] The determining of the mixed similarity based on the feature similarity and the corresponding attribute similarity includes:

[0034] The mixed similarity is determined based on the second feature similarity and the attribute similarity between the target object and the second reference object.

[0035] According to a retrieval method provided by the present invention, the determining the retrieval result based on each of the mixed similarities includes:

[0036] Determine a target mixed similarity greater than or equal to a third preset value among all the mixed similarities;

[0037] The reference objects corresponding to the target mixed similarities are determined as the retrieval results.

[0038] The present invention also provides a retrieval device, comprising:

[0039] A first determination unit is used to determine a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to characterize an attribute of the target object; and a second element in the first confidence matrix corresponding to each first element is used to characterize a confidence of the corresponding attribute;

[0040] A second determining unit, configured to determine, for each reference object in the database, an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0041] a third determining unit, configured to determine a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and a second confidence matrix of the reference object;

[0042] a fourth determining unit, configured to determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0043] The fifth determining unit is used to determine the search result based on the attribute similarities.

[0044] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned retrieval methods is implemented.

[0045] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the retrieval method described above is implemented.

[0046] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned retrieval methods.

[0047] The retrieval method, device, electronic device and storage medium provided by the present invention determine the first attribute value matrix and the first confidence matrix of the target object, characterize the attribute of the target object by each first element in the first attribute value matrix, characterize the confidence of the corresponding attribute by each second element in the first confidence matrix, determine the attribute distance matrix between the target object and the reference object based on the first attribute value matrix of the target object and the second attribute value matrix of the reference object for each reference object in the database, and determine the confidence correlation matrix between the target object and the reference object based on the first confidence matrix of the target object and the second confidence matrix of the reference object, determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix, and then determine the retrieval result in the database that is the same or similar to the target object based on each attribute similarity. It can be seen that the present invention represents the confidence of the attribute value at the corresponding position in the attribute value matrix through each element in the confidence matrix. Based on the confidence correlation matrix and the attribute distance matrix, the attribute similarity between the target object and the reference object can be determined. The attribute similarity can comprehensively reflect the similarity between the attribute values ​​of the target object and the reference object, thereby retrieving the reference object similar to the target object, making the retrieval results richer. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This is one of the flowcharts of the retrieval method provided by the embodiment of the present invention;

[0050] Figure 2 This is the second flow chart of the search method provided by the embodiment of the present invention;

[0051] Figure 3 This is the third flow chart of the search method provided by the embodiment of the present invention;

[0052] Figure 4 This is a fourth flowchart of the search method provided by an embodiment of the present invention;

[0053] Figure 5 is a schematic diagram of the structure of a retrieval device provided by an embodiment of the present invention;

[0054] Figure 6 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Combine the following Figure 1-Figure 4 The retrieval method of the present invention is described. The execution subject of the retrieval method can be an electronic device such as a computer, a server or a server cluster, or a retrieval device set in the electronic device, and the retrieval device can be implemented by software, hardware or a combination of the two. The method can be applied to target retrieval scenarios in the field of security monitoring, for example, face retrieval, body retrieval, or vehicle retrieval and other scenarios.

[0057] Figure 1 is one of the flowcharts of the retrieval method provided by the embodiment of the present invention, such as Figure 1 As shown, the retrieval method includes the following steps:

[0058] Step 101, determine a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; and the second elements corresponding to each first element in the first confidence matrix are used to represent the confidence of the corresponding attribute value.

[0059] The target object may be an object in a picture or video frame, or an object input by a user. The picture or video frame may be obtained based on actual retrieval requirements. For example, the picture or video frame may be obtained by shooting a road monitoring device, and the target object may be a vehicle in the picture or video frame. Each first element in the first attribute value matrix represents the attribute value of each attribute of the target object, for example, the color attribute of the vehicle is white, and the type attribute of the vehicle is automobile. Accordingly, the second element corresponding to each first element in the first confidence matrix is ​​used to represent the confidence of the corresponding attribute value, that is, to represent the degree of trustworthiness of the attribute taking the attribute value. The confidence value range is [0, 1]. The larger the confidence value, the more accurate the attribute value of the attribute, and the smaller the confidence value, the less accurate the attribute value of the attribute.

[0060] For example, depending on different scenarios, the user can choose to manually set the attribute values ​​of each attribute to be compared and the confidence level corresponding to each attribute value, or input the image obtained by the user into a deep learning model, extract each attribute in the image through the deep learning model, and calculate the attribute value corresponding to each attribute and the confidence level corresponding to each attribute value.

[0061] When the attribute value of each attribute of the target object is obtained, each attribute value is used as an element in a first attribute value matrix to obtain a first attribute value matrix, and the confidence corresponding to each attribute value is used as an element in a first confidence matrix to obtain a first confidence matrix.

[0062] Step 102: For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object.

[0063] For example, a large number of pictures and second attribute value matrices and second confidence matrices of reference objects in the pictures are stored in the database. There are many kinds of attributes that have been analyzed by the deep learning model and / or set by the user. When searching, which attributes are used for searching can be set based on the actual scene. If the attribute restriction condition is set manually, the attribute used for the search mainly depends on the user's need for the search, or the user's artificial memory of the target object. If the target object in the picture is analyzed by the deep learning model, the attribute used for the search mainly depends on factors such as the angle and orientation of the target object in the picture. In order to integrate the needs of different scenes, the present invention constructs a calculation model for the attribute similarity between the target object and the reference object, which includes the calculation of the confidence correlation matrix between the target object and the reference object, that is, based on the first confidence matrix of the target object and the second confidence matrix of the reference object in the database, the confidence correlation matrix is ​​determined. Since the essence of each element in the confidence correlation matrix is ​​still to represent the confidence of the attribute value, the value interval is also [0, 1].

[0064] Step 103: Determine an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object.

[0065] By way of example, the computational model also includes calculation of an attribute distance matrix between the target object and the reference object, that is, determining the attribute distance matrix between the target object and the reference object based on a first attribute value matrix of the target object and a second attribute value matrix of the reference object; the elements in the attribute distance matrix indicate whether the corresponding attributes of the target object and the reference object are the same or different, and there are only two results: the same or different. For example, the same attributes can be represented by 1, and the different attributes can be represented by -1.

[0066] Step 104: Determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix.

[0067] Optionally, when the same attribute is represented by 1 and the different attribute is represented by -1, the attribute similarity between the target object and the reference object can be determined based on the following formula (1).

[0068]

[0069] Among them, C nori represents the confidence correlation matrix between the target object and the i-th reference object in the database, A di represents the attribute distance matrix between the target object and the i-th reference object in the database, A di The transposed matrix of Ai Represents the attribute similarity between the target object and the i-th reference object in the database. Based on the attribute distance matrix A di The value of the element in is 1 or -1, and the confidence correlation matrix C nori The value of the element in is [0, 1]. In the above formula (1) The maximum value of is less than or equal to N, so it can be guaranteed that S Ai The value of S is between [0, 1]. Ai The closer the value of is to 1, the greater the attribute similarity between the target object and the i-th reference object. Ai The closer the value of is to 0, the smaller the attribute similarity between the target object and the i-th reference object is. The above formula (1) is simple and easy to implement, with low computational complexity, and introduces the confidence correlation matrix C nori For the attribute distance matrix A di Normalization can obtain a more accurate attribute similarity S Ai , thus making the final retrieval results more accurate.

[0070] It should be noted that the attribute similarity between the target object and the reference object may also be determined based on the cosine similarity calculation method, the confidence correlation matrix and the attribute distance matrix, and the present invention does not limit this.

[0071] Step 105: Determine the search result based on the attribute similarities.

[0072] For example, when the attribute similarity between the target object and each reference object in the database is obtained, each attribute similarity can be compared with a preset similarity threshold T, and the reference objects corresponding to the attribute similarity greater than or equal to the preset similarity threshold T are used as the search results, and the reference objects corresponding to the attribute similarity less than the preset similarity threshold T are filtered, and the search results are output, so that the user can view the reference objects that are the same or similar to the target object. In addition, the specific value of the preset similarity threshold T can be set based on actual conditions. If the requirement for the search accuracy is high, a larger preset similarity threshold T can be set; if the requirement for the search accuracy is low, a smaller preset similarity threshold T can be set, and the present invention does not limit this.

[0073] It should be noted that when the search results include multiple reference objects, the attribute similarity between the reference objects and the target objects can be sorted based on a preset sorting rule, and the sorting results can be output; the preset sorting rule can be from large to small or from small to large, etc., which can be set based on demand, and the present invention is not limited to this.

[0074] The retrieval method provided by the present invention determines the first attribute value matrix and the first confidence matrix of the target object, represents the attribute of the target object through each first element in the first attribute value matrix, represents the confidence of the corresponding attribute through each second element in the first confidence matrix, determines the attribute distance matrix between the target object and the reference object based on the first attribute value matrix of the target object and the second attribute value matrix of the reference object for each reference object in the database, and determines the confidence correlation matrix between the target object and the reference object based on the first confidence matrix of the target object and the second confidence matrix of the reference object, determines the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix, and then determines the retrieval result that is the same or similar to the target object in the database based on each attribute similarity. It can be known that the present invention represents the confidence of the attribute value of the corresponding position in the attribute value matrix through each element in the confidence matrix, and can determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix, and can comprehensively reflect the similarity between each attribute value of the target object and the reference object through the attribute similarity, thereby retrieving the reference object similar to the target object, making the retrieval result richer.

[0075] In one embodiment, the above step 102 determines the confidence correlation matrix between the target object and the reference object based on the first confidence matrix and the second confidence matrix of the reference object, which can be specifically implemented in the following manner:

[0076] For each of the second elements in the first confidence matrix, a correlation operation is performed between the second element and the third element in the second confidence matrix at the same position as the second element to obtain the confidence correlation matrix.

[0077] Specifically, the confidence correlation matrix between the target object and the reference object can be determined based on the following formula (2):

[0078] C nori =C⊙C i (2)

[0079] Among them, C represents the first confidence matrix, C i represents the second confidence matrix of the i-th reference object, C and C i The dimensions are all 1*N, where N is the number of attributes, ⊙ represents related operations, and C nori Represents the confidence correlation matrix between the target object and the i-th reference object. For example, the elements of the first confidence matrix and the second confidence matrix of the i-th reference object at the corresponding positions are multiplied, or the average value is taken.

[0080] In one embodiment, the above step 103 determines the attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object, which can be specifically implemented in the following manner:

[0081] For each of the first elements in the first attribute value matrix, determine the difference between the first element and the fourth element in the second attribute value matrix at the same position as the first element to obtain a difference matrix; perform fixed value processing on each of the fifth elements in the difference matrix to obtain a fixed value result corresponding to each of the fifth elements; the fixed value result is used to characterize whether the attribute of the target object is the same as the attribute of the reference object; and determine the attribute distance matrix based on each of the fixed value results.

[0082] Specifically, determining the attribute distance matrix based on each fixed value result can be achieved in the following two ways:

[0083] The first method is to calculate the attribute distance matrix between the target object and the reference object based on the following formula (3):

[0084] A di1 =1-2*abs(sign(AA i )) (3)

[0085] Among them, A di1 represents the attribute distance matrix between the target object and the i-th reference object, A represents the first attribute value matrix, A irepresents the second attribute value matrix of the i-th reference object, abs() is the absolute value function, sign() is the sign function, A and A i The dimensions of are all 1*N. Since the attribute values ​​of two attributes are either the same or different, sign(AA i ) takes the value of -1, 0, or 1, and sign(AA i ) is taken as the fixed value result, and abs(sign(AA i )) operation, the value is 1 or 0, and then 1-2*abs(sign(AA i ))After operation, A di1 The value of each element in is converted to 1 or -1, 1 indicates that the corresponding attributes of the target object and the reference object are the same, and -1 indicates that the corresponding attributes of the target object and the reference object are different. In this way, the more identical attributes there are, the greater the final attribute similarity will be, making the final retrieval result more accurate. In addition, it is simple and easy to obtain the attribute distance matrix containing 1 or -1 through formula (3), and the calculation complexity is low.

[0086] The second method is to calculate the attribute distance matrix between the target object and the reference object based on the following formula (4):

[0087] A di2 =1-abs(sign(AA i )) (4)

[0088] Among them, A di represents the attribute distance matrix between the target object and the i-th reference object, A represents the first attribute value matrix, A i represents the second attribute value matrix of the i-th reference object, abs() is the absolute value function, sign() is the sign function, A and A i The dimensions of are all 1*N. Since the attribute values ​​of two attributes are either the same or different, sign(AA i ) takes the value of -1, 0, or 1, and sign(AA i ) is taken as the fixed value result, and then 1-abs(sign(AA i ))After operation, A di2 The value of each element in is converted to 1 or 0, 1 indicates that the corresponding attributes of the target object and the reference object are the same, and 0 indicates that the corresponding attributes of the target object and the reference object are different. In this way, the more identical attributes there are, the greater the final attribute similarity will be, and richer retrieval results can be obtained. It is simple and easy to obtain the attribute distance matrix containing 1 or 0 through formula (4), and the calculation complexity is low.

[0089] It should be noted that, when the same attribute is represented by 1 and the different attribute is represented by 0, the attribute similarity between the target object and the reference object can be determined based on the following formula (5).

[0090]

[0091] In the traditional feature value-based retrieval method, when the deep learning model of the feature value analysis system for each reference object in the database is completely consistent, and the deep learning model is consistent with the deep learning model of the feature value analysis system of the target object, it is feasible to directly use the feature value for comparison to obtain the similarity between the target object and the reference object. However, this feature value-based retrieval method has a single usage scenario, and in most cases, the feature value will focus on the overall characteristics of the target, and pay less attention to local attributes (attributes of the target). Therefore, in this embodiment, a hybrid retrieval is performed by combining feature values ​​and attributes, taking into account both global features and local attributes, so that the retrieval results are more accurate than feature value-based retrieval.

[0092] In one embodiment, Figure 2 FIG. 2 is a flow chart of the search method provided by the embodiment of the present invention. Figure 2 As shown, the above step 105 determines the search results based on the attribute similarities, which can be specifically implemented by the following steps:

[0093] Step 1051: Determine a first characteristic value of the target object.

[0094] The eigenvalue is also called a semi-structured feature, which represents a string of numbers (usually a float array) inferred by the model for the target object. For the model, this string of numbers can be used to represent the overall characteristics of the target object.

[0095] For example, the target object is input into the deep learning model of the feature value parsing system to obtain the first feature value of the target object output by the deep learning model. The training process of the deep learning model is as follows: multiple sample objects are input into the initial deep learning model to obtain the predicted feature values ​​corresponding to each sample object output by the initial deep learning model, and the model parameters of the initial deep learning model are optimized based on the predicted feature values ​​and the feature label values ​​corresponding to the sample objects until convergence to obtain a deep learning model.

[0096] Step 1052: For each of the reference objects, determine the feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object.

[0097] For example, the feature similarity between the target object and the reference object can be determined based on the first eigenvalue of the target object and the second eigenvalue of the reference object through algorithms such as cosine distance, Euclidean distance, Manhattan distance, or Chebyshev distance. The same method can be used to calculate the feature similarity between the target object and each reference object in the database. The value of the feature similarity is generally between 0 and 1. The closer to 1, the more likely it is that the target object and the reference object are the same entity target.

[0098] Step 1053: Determine the mixed similarity based on the feature similarity and the corresponding attribute similarity.

[0099] For example, the mixed similarity may be determined based on the following formula (6).

[0100] S Hi =δ*S Ai +(1-δ)*S ei (6)

[0101] Among them, S Hi represents the mixed similarity between the target object and the i-th reference object in the database, S Ai represents the attribute similarity between the target object and the i-th reference object in the database, S ei It represents the feature similarity between the target object and the i-th reference object in the database. δ is the comparison weight, 0<δ<1. Its value depends on the overall quality of the attribute retrieval method and the feature retrieval method. For example, if the retrieval accuracy of the attribute retrieval method is higher than that of the feature retrieval method, then the value of δ is greater than (1-δ). Alternatively, if the user pays more attention to the global features, then the value of δ is less than (1-δ), for example, δ=0.3.

[0102] It should be noted that in S Ai and S ei When no stretching is performed, the above formula (6) can also be converted to S Hi =δ*f(S Ai )+(1-δ)*f(S ei ), where f(S Ai ) indicates that the attribute similarity is stretched equally. This is because when the attribute similarity is converted externally, in order to increase the comparison accuracy of the same target, different stages of stretching will be performed. In order to achieve equal attribute similarity, feature similarity also needs to be stretched. That is, f(S ei ) indicates equal stretching of feature similarity.

[0103] It should be noted that in S Ai After stretching, S eiWithout stretching, the above formula (6) can also be converted to S Hi =δ*S Ai +(1-δ)*f(S ei ); in S Ai Without stretching, S ei After stretching, the above formula (6) can also be converted to S Hi =δ*f(S Ai )+(1-δ)*S ei .

[0104] Step 1054: Determine the search result based on each of the mixed similarities.

[0105] Optionally, among all the mixed similarities, a target mixed similarity greater than or equal to a third preset value is determined; and a reference object corresponding to each of the target mixed similarities is determined as the retrieval result.

[0106] Among them, the third preset value can be set based on actual needs, and the present invention will not be repeated here.

[0107] For example, when the mixed similarity between the target object and the reference objects in the database is obtained, each mixed similarity can be compared with a third preset value, and the reference objects corresponding to the mixed similarities greater than or equal to the third preset value are used as retrieval results, and the corresponding reference objects are sorted and output based on the size of the mixed similarities, so that users can view the reference objects that are the same or similar to the target object; and the reference objects corresponding to the mixed similarities less than the third preset value are filtered.

[0108] In this embodiment, a hybrid search is performed by combining feature values ​​and attributes, taking into account both global features and local attributes, so that the search results are more accurate than those based on feature values.

[0109] In the traditional method of searching using feature values, there are two search methods based on the amount of data in the database and the hardware capabilities of the server where the search is performed. In the first search method, when the amount of data in the database is small, a full comparison traversal method is usually used during the search and comparison to compare the feature values ​​of the target object with the feature values ​​of all reference objects in the database, and then filter and sort the search results according to the feature similarity. In the second search method, when the amount of data in the database is large, all feature values ​​in the database are usually clustered first, and then the feature values ​​of the target object are compared in layers to reduce the number of comparisons and improve the comparison efficiency. In response to these two search methods, the present invention respectively establishes two hybrid search processes.

[0110] In one embodiment, Figure 3FIG. 3 is a flow chart of the retrieval method provided by an embodiment of the present invention. Figure 3 As shown, the above steps 1052 and 1053 can be implemented by the following steps:

[0111] Step 10521: When the number of the reference objects is less than a preset number, delete all attribute similarities whose attribute similarities are less than a first preset value, and determine the reference objects corresponding to the remaining attribute similarities as first reference objects.

[0112] The preset number can be determined based on the performance of the electronic device, for example, the preset number is 100; the first preset value S et It can be converted based on a preset similarity threshold T set by the user, for example, the first preset value S et =T+δ-1, δ is the above-mentioned comparison weight.

[0113] For example, since the time consumed for comparing a single feature value is much longer than the time consumed for comparing a single attribute value, when the number of reference objects in the database is less than a preset number, you can first search based on the attribute value, that is, determine the attribute similarity between the target object and each reference object in the database, and then find out the attribute similarities that are less than a first preset value from all the attribute similarities, eliminate the attribute similarities that are less than the first preset value, and obtain the remaining attribute similarities. The reference objects corresponding to each remaining attribute similarity are all determined as the first reference objects, thereby filtering the reference objects in the database.

[0114] Step 10522: For each of the first reference objects, determine a first feature similarity between the target object and the first reference object based on the first feature value and the second feature value of the first reference object.

[0115] For example, for each first reference object, the first feature similarity between the target object and the first reference object is determined based on the first eigenvalue of the target object and the second eigenvalue of the first reference object through algorithms such as cosine distance, Euclidean distance, Manhattan distance, or Chebyshev distance. The same method can be used to calculate the first feature similarity between the target object and each first reference object.

[0116] Step 10531: Determine the mixed similarity based on the first feature similarity and the corresponding remaining attribute similarities.

[0117] For example, the mixed similarity can be obtained by weighted summing the first feature similarity and the corresponding remaining attribute similarities. Specifically, the mixed similarity can be determined based on the following formula (7).

[0118] S Hj =δ*S Aj+(1-δ)*S ej (7)

[0119] Among them, S Hj represents the mixed similarity between the target object and the jth first reference object in the database, S Aj represents the attribute similarity between the target object and the jth first reference object in the database, S ej It represents the feature similarity between the target object and the j-th first reference object in the database. δ is the comparison weight, 0<δ<1. Its value depends on the overall quality of the attribute retrieval method and the feature retrieval method. For example, if the retrieval accuracy of the attribute retrieval method is higher than that of the feature retrieval method, then the value of δ is greater than (1-δ). Alternatively, if the user pays more attention to the global features, then the value of δ is less than (1-δ), for example, δ=0.3.

[0120] In this embodiment, when the number of reference objects in the database is less than the preset number, the reference objects in the database are filtered based on the attribute value, and finally the search result is obtained based on the mixed similarity between the target object and each first reference object. Since the feature comparison is based on the filtered first reference object, the calculation amount of feature comparison is reduced, thereby reducing the search time and improving the search efficiency.

[0121] In one embodiment, Figure 4 FIG. 4 is a flowchart of a search method according to an embodiment of the present invention. Figure 4 As shown, the above steps 1052 and 1053 can be specifically implemented by the following steps:

[0122] Step 10523: When the number of the reference objects is greater than or equal to the preset number, cluster the second eigenvalues ​​of all the reference objects to obtain at least two clusters.

[0123] Among them, the second preset value S et1 It can be converted based on the preset similarity threshold T set by the user, for example, the second preset value S et1 =T-δ, δ is the comparison weight mentioned above.

[0124] For example, when the number of reference objects in the database is greater than or equal to a preset number, the second eigenvalues ​​of all reference objects in the database are clustered based on a clustering algorithm to obtain at least two clusters, and the second eigenvalues ​​of the reference objects in each cluster are the same or similar.

[0125] Step 10524: For each of the clusters, determine a second feature similarity between the target object and a reference object in the cluster based on the first feature value and a second feature value corresponding to the cluster.

[0126] The second eigenvalue corresponding to the cluster may be an average value of the second eigenvalues ​​of all reference objects in the cluster, or may be the second eigenvalue of any reference object in the cluster.

[0127] For example, for each cluster obtained by clustering, the second feature similarity between the target object and the reference object in the cluster is determined based on the first eigenvalue of the target object and the second eigenvalue corresponding to the cluster through algorithms such as cosine distance, Euclidean distance, Manhattan distance, or Chebyshev distance. The same method can be used to calculate the second feature similarity between the target object and the reference objects in each cluster.

[0128] Step 10525: Delete all the second feature similarities that are less than a second preset value among the second feature similarities, and determine the clusters corresponding to the remaining second feature similarities as target clusters.

[0129] For example, second feature similarities less than a second preset value are found from all second feature similarities, and the second feature similarities less than the second preset value are eliminated to obtain the remaining second feature similarities, and the clusters corresponding to the remaining second feature similarities are determined as target clusters, thereby achieving unified filtering of reference objects included in the cluster.

[0130] Step 10526: For each second reference object in the target cluster, determine a second feature similarity between the target object and the second reference object based on the first feature value and the second feature value of the second reference object.

[0131] For example, for each second reference object in the target cluster, the second feature similarity between the target object and the second reference object is determined based on the first eigenvalue of the target object and the second eigenvalue of the second reference object through algorithms such as cosine distance, Euclidean distance, Manhattan distance, or Chebyshev distance. Using the same method, the second feature similarity between the target object and each second reference object in the target cluster can be calculated.

[0132] Step 10532: Determine the mixed similarity based on the second feature similarity and the attribute similarity between the target object and the second reference object.

[0133] For example, the mixed similarity can be obtained by weighted summing the second feature similarity and the attribute similarity between the target object and the second reference object. Specifically, the mixed similarity can be determined based on the following formula (8).

[0134] S Hk =δ*S Ak +(1-δ)*S ek (8)

[0135] Among them, SHk represents the mixed similarity between the target object and the kth second reference object in the target cluster, S Ak represents the attribute similarity between the target object and the kth second reference object in the target cluster, S ek It represents the feature similarity between the target object and the kth second reference object in the target cluster. δ is the comparison weight, 0<δ<1. Its value depends on the overall quality of the attribute retrieval method and the feature retrieval method. For example, if the retrieval accuracy of the attribute retrieval method is higher than that of the feature retrieval method, then the value of δ is greater than (1-δ). Alternatively, if the user pays more attention to the global features, then the value of δ is less than (1-δ), for example, δ=0.3.

[0136] In this embodiment, when the number of reference objects in the database is greater than or equal to a preset number, the feature values ​​of the reference objects in the database are first clustered, and then the clusters with a second feature similarity less than a second preset value are eliminated, and finally the retrieval results are obtained based on the mixed similarity between the target object and the second reference object in each target cluster, thereby reducing the number of feature comparisons and improving the comparison efficiency, thereby reducing the retrieval time and improving the retrieval efficiency.

[0137] The retrieval device provided by the present invention is described below. The retrieval device described below and the retrieval method described above can be referenced to each other.

[0138] Figure 5 is a schematic diagram of the structure of a retrieval device provided by an embodiment of the present invention, such as Figure 5 As shown, the search device 500 includes a first determination unit 501, a second determination unit 502, a third determination unit 503, a fourth determination unit 504 and a fifth determination unit 505; wherein:

[0139] A first determination unit 501 is used to determine a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to represent an attribute of the target object; and a second element in the first confidence matrix corresponding to each first element is used to represent a confidence of the corresponding attribute;

[0140] A second determining unit 502 is configured to determine, for each reference object in the database, a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and a second confidence matrix of the reference object;

[0141] A third determining unit 503 is used to determine an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0142] A fourth determining unit 504 is used to determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0143] The fifth determining unit 505 is configured to determine the search result based on the attribute similarities.

[0144] The retrieval device provided by the present invention determines the first attribute value matrix and the first confidence matrix of the target object, represents the attribute of the target object through each first element in the first attribute value matrix, represents the confidence of the corresponding attribute through each second element in the first confidence matrix, and for each reference object in the database, determines the attribute distance matrix between the target object and the reference object based on the first attribute value matrix of the target object and the second attribute value matrix of the reference object, and determines the confidence correlation matrix between the target object and the reference object based on the first confidence matrix of the target object and the second confidence matrix of the reference object, determines the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix, and then determines the retrieval result that is the same or similar to the target object in the database based on each attribute similarity. It can be known that the present invention represents the confidence of the attribute value of the corresponding position in the attribute value matrix through each element in the confidence matrix, and can determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix, and can comprehensively reflect the similarity between each attribute value of the target object and the reference object through the attribute similarity, thereby retrieving the reference object similar to the target object, making the retrieval result richer.

[0145] Based on any of the foregoing embodiments, the second determining unit 502 is specifically configured to:

[0146] For each of the second elements in the first confidence matrix, a correlation operation is performed between the second element and the third element in the second confidence matrix at the same position as the second element to obtain the confidence correlation matrix.

[0147] Based on any of the foregoing embodiments, the third determining unit 503 is specifically configured to:

[0148] For each of the first elements in the first attribute value matrix, determine a difference between the first element and a fourth element in the second attribute value matrix at the same position as the first element, to obtain a difference matrix;

[0149] Performing fixed value processing on each fifth element in the difference matrix to obtain a fixed value result corresponding to each fifth element; the fixed value result is used to indicate whether the attribute of the target object is the same as the attribute of the reference object;

[0150] The attribute distance matrix is ​​determined based on each of the determination results.

[0151] Based on any of the foregoing embodiments, the fifth determining unit 505 is specifically configured to:

[0152] Determining a first characteristic value of the target object;

[0153] For each of the reference objects, determining a feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object;

[0154] Determining a mixed similarity based on the feature similarity and the corresponding attribute similarity;

[0155] The search result is determined based on each of the mixed similarities.

[0156] Based on any of the foregoing embodiments, the fifth determining unit 505 is further specifically configured to:

[0157] In the case that the number of the reference objects is less than a preset number, deleting the attribute similarities less than a first preset value among all the attribute similarities, and determining the reference objects corresponding to the remaining attribute similarities as the first reference objects;

[0158] For each of the first reference objects, determining a first feature similarity between the target object and the first reference object based on the first feature value and a second feature value of the first reference object;

[0159] The mixed similarity is determined based on the first feature similarity and the corresponding remaining attribute similarities.

[0160] Based on any of the foregoing embodiments, the fifth determining unit 505 is further specifically configured to:

[0161] When the number of the reference objects is greater than or equal to the preset number, clustering the second eigenvalues ​​of all the reference objects to obtain at least two clusters;

[0162] For each of the clusters, determining a second feature similarity between the target object and a reference object in the cluster based on the first feature value and a second feature value corresponding to the cluster;

[0163] Deleting all the second feature similarities that are less than a second preset value among the second feature similarities, and determining clusters corresponding to the remaining second feature similarities as target clusters;

[0164] For each second reference object in the target cluster, determining a second feature similarity between the target object and the second reference object based on the first feature value and the second feature value of the second reference object;

[0165] The mixed similarity is determined based on the second feature similarity and the attribute similarity between the target object and the second reference object.

[0166] Based on any of the foregoing embodiments, the fifth determining unit 505 is further specifically configured to:

[0167] Determine a target mixed similarity greater than or equal to a third preset value among all the mixed similarities;

[0168] The reference objects corresponding to the target mixed similarities are determined as the retrieval results.

[0169] Figure 6 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the retrieval method, which includes: determining a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; the second element corresponding to each first element in the first confidence matrix is ​​used to represent the confidence of the corresponding attribute value;

[0170] For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object;

[0171] Determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0172] Determining attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0173] The search result is determined based on the similarity of each attribute.

[0174] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0175] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the retrieval method provided by the above methods, the method comprising: determining a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; the second element corresponding to each first element in the first confidence matrix is ​​used to represent the confidence of the corresponding attribute value;

[0176] For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object;

[0177] Determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0178] Determining attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0179] The search result is determined based on the similarity of each attribute.

[0180] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the retrieval method provided by the above methods, the method comprising: determining a first attribute value matrix and a first confidence matrix of a target object; each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; and the second elements corresponding to each first element in the first confidence matrix are used to represent the confidence of the corresponding attribute value;

[0181] For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object;

[0182] Determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object;

[0183] Determining attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix;

[0184] The search result is determined based on the similarity of each attribute.

[0185] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A search method, characterized in that: include: Determine a first attribute value matrix and a first confidence matrix of the target object; Each first element in the first attribute value matrix is ​​used to represent the attribute value of each attribute in the target object; The second elements corresponding to the first elements in the first confidence matrix are used to represent the confidence of the corresponding attribute value; For each reference object in the database, based on the first confidence matrix and the second confidence matrix of the reference object, determine a confidence correlation matrix between the target object and the reference object; Determining an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and the second attribute value matrix of the reference object; Determining attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix; The search result is determined based on the similarity of each attribute.

2. The search method according to claim 1, characterized in that: The step of determining a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and the second confidence matrix of the reference object comprises: For each of the second elements in the first confidence matrix, a correlation operation is performed between the second element and the third element in the second confidence matrix at the same position as the second element to obtain the confidence correlation matrix.

3. The search method according to claim 1, characterized in that: The determining, based on the first attribute value matrix and the second attribute value matrix of the reference object, an attribute distance matrix between the target object and the reference object comprises: For each of the first elements in the first attribute value matrix, determine a difference between the first element and a fourth element in the second attribute value matrix at the same position as the first element, to obtain a difference matrix; Performing fixed value processing on each fifth element in the difference matrix to obtain a fixed value result corresponding to each fifth element; the fixed value result is used to indicate whether the attribute of the target object is the same as the attribute of the reference object; The attribute distance matrix is ​​determined based on each of the determination results.

4. The search method according to claim 1, characterized in that: The determining of the search result based on the attribute similarity comprises: Determining a first characteristic value of the target object; For each of the reference objects, determining a feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object; Determining a mixed similarity based on the feature similarity and the corresponding attribute similarity; The search result is determined based on each of the mixed similarities.

5. The search method according to claim 4, characterized in that: The determining, for each of the reference objects, a feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object, includes: In the case that the number of the reference objects is less than a preset number, deleting the attribute similarities less than a first preset value among all the attribute similarities, and determining the reference objects corresponding to the remaining attribute similarities as the first reference objects; For each of the first reference objects, determining a first feature similarity between the target object and the first reference object based on the first feature value and a second feature value of the first reference object; The determining of the mixed similarity based on the feature similarity and the corresponding attribute similarity includes: The mixed similarity is determined based on the first feature similarity and the corresponding remaining attribute similarities.

6. The search method according to claim 4, characterized in that: The determining, for each of the reference objects, a feature similarity between the target object and the reference object based on the first feature value and the second feature value of the reference object, includes: When the number of the reference objects is greater than or equal to a preset number, clustering the second eigenvalues ​​of all the reference objects to obtain at least two clusters; For each of the clusters, determining a second feature similarity between the target object and a reference object in the cluster based on the first feature value and a second feature value corresponding to the cluster; Deleting all the second feature similarities that are less than a second preset value among the second feature similarities, and determining clusters corresponding to the remaining second feature similarities as target clusters; For each second reference object in the target cluster, determining a second feature similarity between the target object and the second reference object based on the first feature value and the second feature value of the second reference object; The determining of the mixed similarity based on the feature similarity and the corresponding attribute similarity includes: The mixed similarity is determined based on the second feature similarity and the attribute similarity between the target object and the second reference object.

7. The search method according to any one of claims 4 to 6, characterized in that: The determining the search result based on each of the mixed similarities comprises: Determine a target mixed similarity greater than or equal to a third preset value among all the mixed similarities; The reference objects corresponding to the target mixed similarities are determined as the retrieval results.

8. A search device, characterized in that: include: A first determining unit, used to determine a first attribute value matrix and a first confidence matrix of the target object; Each first element in the first attribute value matrix is ​​used to represent the attribute of the target object; The second elements corresponding to the first elements in the first confidence matrix are used to represent the confidence of the corresponding attribute; A second determination unit, configured to determine, for each reference object in the database, a confidence correlation matrix between the target object and the reference object based on the first confidence matrix and a second confidence matrix of the reference object; a third determining unit, configured to determine an attribute distance matrix between the target object and the reference object based on the first attribute value matrix and a second attribute value matrix of the reference object; a fourth determining unit, configured to determine the attribute similarity between the target object and the reference object based on the confidence correlation matrix and the attribute distance matrix; The fifth determining unit is used to determine the search result based on the attribute similarities.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the search method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the search method according to any one of claims 1 to 7 is implemented.

Images