Archive merging method and apparatus, computer device, and computer readable storage medium

Abstract

The present application relates to the technical field of computer, specifically relates to a kind of file merging method, device, computer equipment and computer readable storage medium, to solve the problem of improving the accuracy of file.For this purpose, the method of the present application includes determining the first nearest neighbor coincidence degree of first data library (the data library formed by the first to be detected file and the interference set) in the first to be detected file, the second nearest neighbor coincidence degree of second data library (the data library formed by the first to be detected file, the second to be detected file and the interference set) in the first to be detected file, according to the first and second nearest neighbor coincidence degree, determine whether to merge the first to be detected file and the second to be detected file file.Through the above-mentioned mode, the correlation between the file data in different to-be-detected files can be fully tapped, so that the to-be-detected files belonging to the same target object can be merged to the greatest extent, and the accuracy of file is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an archive merging method and device, computer equipment and computer readable storage medium. BACKGROUND

[0002] In order to facilitate the storage and management of object data (such as a face image) of a target object (such as a pedestrian), an archive is usually established for each target object to store all object data of the target object. After obtaining a new object data, the new object data and the object data in the archive of the existing target object can be clustered to store the new object data in the archive of the existing target object or establish a new archive for a new target object based on the new object data. In order to improve the efficiency of clustering, an incremental clustering algorithm is usually used for clustering. However, when the incremental clustering algorithm is used for clustering, the existing archives may be merged or split. In order to balance the relationship between clustering precision and recall, multiple archives for the same target object are often established, which cannot store all object data of the target object through one archive, and reduces the accuracy of the archive.

[0003] Therefore, there is a need in the art for a new technical solution to solve the above problems. SUMMARY

[0004] In order to overcome the above defects, the present application is proposed to provide an archive merging method, device, computer equipment and computer readable storage medium to solve or at least partially solve the technical problem of how to avoid establishing multiple archives for the same target object (such as a pedestrian) and improve the accuracy of the archive.

[0005] In a first aspect, an archive merging method is provided, the method comprising:

[0006] forming a first data base according to the first archive data in the first to-be-detected archive and the interference data in the preset interference set, retrieving first base nearest neighbor data of each first archive data in the first data base, and determining a first base nearest neighbor coincidence degree of the first to-be-detected archive in the first data base according to the coincidence degree between the first base nearest neighbor data corresponding to different first archive data.

[0007] form a second data base according to the first archive data in the first to-be-detected archive, second archive data in the second to-be-detected archive and interference data in the preset interference set, search for second base nearest neighbor data of each first archive data in the second data base, and determine a second base nearest neighbor coincidence degree of the first to-be-detected archive in the second data base according to coincidence degrees between the second base nearest neighbor data corresponding to different first archive data respectively;

[0008] determine whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first base nearest neighbor coincidence degree and the second base nearest neighbor coincidence degree.

[0009] In one technical solution of the archive merging method, the step of "determining whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first base nearest neighbor coincidence degree and the second base nearest neighbor coincidence degree" specifically includes:

[0010] determining a coincidence degree deviation between the second base nearest neighbor coincidence degree and the first base nearest neighbor coincidence degree;

[0011] selectively performing archive merging on the first to-be-detected archive and the second to-be-detected archive according to a comparison result of the coincidence degree deviation and a preset deviation threshold, specifically including:

[0012] if the coincidence degree deviation is greater than or equal to the preset deviation threshold, performing archive merging;

[0013] if the coincidence degree deviation is less than the preset deviation threshold, not performing archive merging.

[0014] In one technical solution of the archive merging method, the step of "selectively performing archive merging on the first to-be-detected archive and the second to-be-detected archive according to a comparison result of the coincidence degree deviation and a preset deviation threshold" further includes:

[0015] forming a third data base according to the second archive data in the second to-be-detected archive and interference data in the preset interference set, searching for third base nearest neighbor data of each second archive data in the third data base, and determining a third base nearest neighbor coincidence degree of the second to-be-detected archive in the third data base according to coincidence degrees between the third base nearest neighbor data corresponding to different second archive data respectively;

[0016] judging whether the coincidence degree deviation is greater than or equal to the preset deviation threshold and whether the first base nearest neighbor coincidence degree and the third base nearest neighbor coincidence degree are greater than or equal to preset coincidence degree thresholds respectively;

[0017] If yes, then file merging is performed;

[0018] If no, then file merging is not performed.

[0019] In one of the technical solutions of the above file merging method, the method further comprises determining the first data library nearest neighbor coincidence degree or the second data library nearest neighbor coincidence degree or the third data library nearest neighbor coincidence degree respectively by the following manner:

[0020] According to the coincidence degree between the first data library nearest neighbor data or the second data library nearest neighbor data or the third data library nearest neighbor data, and by the following formula, the first data library nearest neighbor coincidence degree or the second data library nearest neighbor coincidence degree or the third data library nearest neighbor coincidence degree is determined respectively:

[0021]

[0022] Wherein,

[0023] When D base represents the first data library, S represents the first to-be-detected file, sn(S1, D base ) represents the first data library nearest neighbor coincidence degree, n represents the total number of first file data in the first to-be-detected file, psn ij represents the coincidence degree between the first data library nearest neighbor data corresponding to the i-th first file data and the j-th first file data in the first to-be-detected file.

[0024] When D base represents the second data library, S represents the first to-be-detected file, sn(S1, D base ) represents the second data library nearest neighbor coincidence degree, n represents the total number of first file data in the first to-be-detected file, psn ij represents the coincidence degree between the second data library nearest neighbor data corresponding to the i-th first file data and the j-th first file data in the first to-be-detected file.

[0025] When D base represents the third data library, S represents the second to-be-detected file, sn(S1, D base ) represents the third data library nearest neighbor coincidence degree, n represents the total number of second file data in the second to-be-detected file, psn ij represents the coincidence degree between the third data library nearest neighbor data corresponding to the i-th second file data and the j-th second file data in the second to-be-detected file.

[0026] In one of the technical solutions of the above archive merging method, the method further comprises calculating the coincidence degree between the first nearest neighbor data of the first database, the second nearest neighbor data of the second database, or the third nearest neighbor data of the third database respectively by the following formula:

[0027] psn ij = N(n i (k)∩n j (k))

[0028] wherein N represents a counting function;

[0029] psn ij represents the coincidence degree between the first nearest neighbor data corresponding to the i th first archive data and the j th first archive data in the first to-be-detected archive, n i (k) and n j (k) represent the k first nearest neighbor data corresponding to the i th first archive data and the j th first archive data in the first to-be-detected archive respectively, k > 1;

[0030] psn ij represents the coincidence degree between the second nearest neighbor data corresponding to the i th first archive data and the j th first archive data in the first to-be-detected archive, n i (k) and n j (k) represent the k second nearest neighbor data corresponding to the i th first archive data and the j th first archive data in the first to-be-detected archive respectively, k > 1;

[0031] psn ij represents the coincidence degree between the third nearest neighbor data corresponding to the i th second archive data and the j th second archive data in the second to-be-detected archive, n i (k) and n j (k) represent the k third nearest neighbor data corresponding to the i th second archive data and the j th second archive data in the second to-be-detected archive respectively, k > 1.

[0032] In one of the technical solutions of the above archive merging method, the method further comprises retrieving the first nearest neighbor data, the second nearest neighbor data, or the third nearest neighbor data respectively by the following method:

[0033] The initial nearest neighbor data of the to-be-detected archive data is retrieved in a to-be-detected database by using a nearest neighbor algorithm;

[0034] The initial nearest neighbor data is filtered according to the similarity between the to-be-detected archive data and each initial nearest neighbor data, to obtain the final nearest neighbor data;

[0035] wherein,

[0036] the initial library nearest neighbor data is initial first library nearest neighbor data when the database to be searched is the first data library and the archive data to be searched is first archive data, and the final library nearest neighbor data is final first library nearest neighbor data;

[0037] the initial library nearest neighbor data is initial second library nearest neighbor data when the database to be searched is the second data library and the archive data to be searched is first archive data, and the final library nearest neighbor data is final second library nearest neighbor data;

[0038] the initial library nearest neighbor data is initial third library nearest neighbor data when the database to be searched is the third data library and the archive data to be searched is second archive data, and the final library nearest neighbor data is final third library nearest neighbor data.

[0039] In one of the technical solutions of the archive merging method, the total number of the second archive data in the second archive to be detected is greater than the total number of the first archive data in the first archive to be detected.

[0040] In a second aspect, an archive merging device is provided, and the device comprises:

[0041] a first library nearest neighbor coincidence degree determining module configured to form a first data library according to first archive data in a first archive to be detected and interference data in a preset interference set, search for first library nearest neighbor data of each of the first archive data in the first data library, and determine a first library nearest neighbor coincidence degree of the first archive to be detected in the first data library according to coincidence degrees between the first library nearest neighbor data corresponding to different first archive data respectively;

[0042] a second library nearest neighbor coincidence degree determining module configured to form a second data library according to the first archive data in the first archive to be detected, second archive data in a second archive to be detected, and the interference data in the preset interference set, search for second library nearest neighbor data of each of the first archive data in the second data library, and determine a second library nearest neighbor coincidence degree of the first archive to be detected in the second data library according to coincidence degrees between the second library nearest neighbor data corresponding to different first archive data respectively;

[0043] an archive merging module configured to determine whether to perform archive merging on the first archive to be detected and the second archive to be detected according to the first library nearest neighbor coincidence degree and the second library nearest neighbor coincidence degree.

[0044] In a third aspect, a computer device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the profile merging method according to any one of the technical solutions of the profile merging method.

[0045] In a fourth aspect, a computer readable storage medium is provided, wherein a plurality of program codes are stored, the program codes being adapted to be loaded and run by a processor to perform the profile merging method according to any one of the technical solutions of the profile merging method.

[0046] The one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0047] In the implementation of the technical solutions of the present application, a first data base can be formed according to the first archive data in the first to-be-detected archive and the interference data in the preset interference set, the first base nearest neighbor data of each first archive data can be searched in the first data base, and the first base nearest neighbor coincidence degree can be determined according to the coincidence degrees between the first base nearest neighbor data corresponding to different first archive data. The greater the first base nearest neighbor coincidence degree is, the higher the correlation between all the first archive data in the first data base is, and vice versa. Similarly, a second data base can be formed according to the first archive data in the first to-be-detected archive, the second archive data in the second to-be-detected archive and the interference data in the interference set, the second base nearest neighbor data of each first archive data can be searched in the second data base, and the second base nearest neighbor coincidence degree can be determined according to the coincidence degrees between the second base nearest neighbor data corresponding to different first archive data. The greater the second base nearest neighbor coincidence degree is, the higher the correlation between all the first archive data in the second data base is, and vice versa.

[0048] After the first base nearest neighbor coincidence degree and the second base nearest neighbor coincidence degree of the first to-be-detected archive are determined, whether to perform the profile merging of the first to-be-detected archive and the second to-be-detected archive can be determined according to the first base nearest neighbor coincidence degree and the second base nearest neighbor coincidence degree. Specifically, if the second base nearest neighbor coincidence degree is greater than the first base nearest neighbor coincidence degree, it indicates that in the second data base, in addition to the first archive data, there are other data similar to the first archive data. Since the first data base also contains the interference data in the interference set, it can be determined that these similar other data are the second archive data in the second to-be-detected archive, and therefore, the profile merging of the first to-be-detected archive and the second to-be-detected archive can be performed.

[0049] By the above manner, the correlation between the archive data in different to-be-detected archives can be fully tapped, so that the to-be-detected archives belonging to the same target object (such as a pedestrian) can be merged together to the greatest extent, and multiple archives for the same target object can be avoided, thereby improving the archive accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0050] The disclosure of the present application will become more apparent with the following description of the drawings. It is readily understood by those skilled in the art that the drawings are only for the purpose of illustration and are not intended to limit the scope of protection of the present application. Among them:

[0051] Figure 1 is the main step flowchart of the archive merging method according to an embodiment of the present application;

[0052] Figure 2 is the main step flowchart of the method for determining whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first base library nearest neighbor coincidence degree and the second base library nearest neighbor coincidence degree according to an embodiment of the present application;

[0053] Figure 3 is the main step flowchart of the method for determining whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first base library nearest neighbor coincidence degree and the second base library nearest neighbor coincidence degree according to another embodiment of the present application;

[0054] Figure 4 is the main structure block diagram of the archive merging device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] Some embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the scope of protection of the present application.

[0056] In the description of the present application, "module" and "processor" can include hardware, software or a combination of both. A module can include hardware circuit, various suitable sensors, communication port, memory, and can also include software part such as program code, and can be a combination of software and hardware. The processor can be a central processor, microprocessor, image processor, digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor can be implemented in software, hardware or a combination of both. The non-transitory computer readable storage medium includes any suitable medium that can store program code, such as magnetic disk, hard disk, optical disk, flash memory, read-only memory, random access memory, etc.

[0057] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a file merging method according to an embodiment of the present invention. Figure 1 As shown, the file merging method in this embodiment of the invention mainly includes the following steps S101 to S105.

[0058] Step S101: Form a first data base based on the first file data in the first file to be detected and the interference data in the preset interference set.

[0059] Interference data refers to data of the same data type as the primary data, but not belonging to the same target object. For example, if the primary data is a facial image of target object A, then interference data could be facial images of other target objects such as B, C, D, and E.

[0060] The total number of interfering data is much greater than the total number of the first file data. For example, the quantity deviation between the interfering data and the first file data can be greater than or equal to a preset quantity deviation threshold. Those skilled in the art can flexibly set the specific value of the quantity deviation threshold according to actual needs, as long as the total number of interfering data is much greater than the total number of the first file data.

[0061] Step S102: Retrieve the nearest neighbor data of the first archive data in the first data base. Based on the overlap between the nearest neighbor data of the first archive data corresponding to each different first archive data, determine the overlap of the nearest neighbor data of the first archive data in the first data base.

[0062] The greater the nearest neighbor overlap in the first database, the higher the correlation among all first-file data in the first database; conversely, the lower the nearest neighbor overlap, the lower the correlation among all first-file data in the first database.

[0063] In this embodiment, a conventional nearest neighbor algorithm in the field of data retrieval technology can be used to retrieve the nearest neighbor data of each first file data in the first data base. For example, the K-Nearest Neighbor (KNN) algorithm can be used to retrieve the nearest neighbor data of each first file data in the first data base. This embodiment of the invention does not specifically limit the nearest neighbor algorithm, as long as it can retrieve multiple data that are most similar to each first file data from the first data base.

[0064] Further, in some embodiments, in order to improve the retrieval accuracy of the nearest neighbor data, a nearest neighbor algorithm can be used first to retrieve the initial first database nearest neighbor data of the first profile data in the first database, and then the initial first database nearest neighbor data is filtered according to the similarity between the first profile data and each initial first database nearest neighbor data to obtain the final first database nearest neighbor data. Specifically, the first database nearest neighbor data with a similarity greater than a preset similarity threshold can be selected as the final first database nearest neighbor data. The specific value of the preset similarity threshold can be flexibly set by those skilled in the art according to actual needs, and the present embodiment does not make specific limitations thereto.

[0065] Step S103: Forming a second database according to the first profile data in the first to-be-detected profile, the second profile data in the second to-be-detected profile, and the interference data in the preset interference set.

[0066] It should be noted that the preset interference set in step S103 is completely the same as the preset interference set in the aforementioned step S101. Moreover, the interference data in the preset interference set is also the same data type as the second profile data, but does not belong to the same target object.

[0067] Step S104: Retrieving the second database nearest neighbor data of each first profile data in the second database, and determining the second database nearest neighbor coincidence degree of the first to-be-detected profile in the second database according to the coincidence degree between the second database nearest neighbor data corresponding to different first profile data.

[0068] The greater the second database nearest neighbor coincidence degree is, the higher the correlation between all the first profile data in the second database is, and vice versa.

[0069] In the present embodiment, the conventional nearest neighbor algorithm in the data retrieval technical field can also be used to retrieve the second database nearest neighbor data of each first profile data in the second database, for example, the K- nearest neighbor algorithm can still be used for retrieval.

[0070] Further, in some embodiments, in order to improve the retrieval accuracy of the nearest neighbor data, the nearest neighbor algorithm can be used first to retrieve the initial second database nearest neighbor data of the first profile data in the second database, and then the initial second database nearest neighbor data can be filtered according to the similarity between the first profile data and each initial second database nearest neighbor data to obtain the final second database nearest neighbor data. Specifically, the second database nearest neighbor data with a similarity greater than a preset similarity threshold can be selected as the final second database nearest neighbor data. The person skilled in the art can flexibly set the specific value of the preset similarity threshold according to the actual needs, and the present embodiment does not make specific limitations.

[0071] Step S105: determining whether to perform profile merging on the first to-be-detected profile and the second to-be-detected profile according to the first database nearest neighbor coincidence degree and the second database nearest neighbor coincidence degree.

[0072] Since the greater the first database nearest neighbor coincidence degree is, the higher the correlation between all the first profile data in the first database is, and the greater the second database nearest neighbor coincidence degree is, the higher the correlation between all the first profile data in the second database is, if the second database nearest neighbor coincidence degree is greater than the first database nearest neighbor coincidence degree, it indicates that in addition to the first profile data, there are other data similar to the first profile data in the second database. Further, since the first database also contains the interference data in the interference set, it can be determined that these similar other data are the second profile data in the second to-be-detected profile, and therefore, in the case where the second database nearest neighbor coincidence degree is greater than the first database nearest neighbor coincidence degree, the first to-be-detected profile and the second to-be-detected profile can be profile-merged. Otherwise, the first to-be-detected profile and the second to-be-detected profile are not profile-merged.

[0073] Through the method described in the above steps S101 to S105, the correlation between the profile data in different to-be-detected profiles can be fully mined, so that the to-be-detected profiles belonging to the same target object (such as a pedestrian) can be merged to the greatest extent, avoiding establishing multiple profiles for the same target object, and finally improving the accuracy of the profile.

[0074] The above steps S102, S104 and S105 will be further described below. First, the above step S102 will be described.

[0075] In some embodiments of the above step S102, the coincidence degree between the first database nearest neighbor data corresponding to each of the different first profile data can be quickly and accurately calculated by the following formula (1), so as to improve the processing efficiency and accuracy of profile merging.

[0076] psn ij= N(n i (k)∩n j (k)) (1)

[0077] The meanings of the parameters in formula (1) are as follows:

[0078] N represents a counting function, and the counting function can be used to obtain the number of the first archive data in the intersection of n i (k) and n j (k). psn ij represents the coincidence degree between the first nearest neighbor data of the first database corresponding to the i-th first archive data and the j-th first archive data in the first to-be-detected archive, n i (k) and n j (k) respectively represent the k first nearest neighbor data of the first database corresponding to the i-th first archive data and the j-th first archive data in the first to-be-detected archive, k>1.

[0079] Through the above formula (1), the number of coincidences in the k first nearest neighbor data of the first database corresponding to the i-th first archive data and the j-th first archive data can be used as the coincidence degree.

[0080] In some other embodiments of the above step S102, the first nearest neighbor coincidence degree of the first to-be-detected archive in the first data library can be accurately determined according to the first nearest neighbor data of the first database and through the following formula (2), so as to further improve the accuracy of the archive merging.

[0081]

[0082] The meanings of the parameters in formula (2) are as follows:

[0083] D base represents the first data library, S represents the first to-be-detected archive, and sn(S1, D base ) represents the first nearest neighbor coincidence degree of the first to-be-detected archive in the first data library, n represents the total number of the first archive data in the first to-be-detected archive, and psn ij represents the coincidence degree between the first nearest neighbor data of the first database corresponding to the i-th first archive data and the j-th first archive data in the first to-be-detected archive. In some embodiments, the coincidence degree between the first nearest neighbor data can be calculated through the above formula (1).

[0084] The above is a further description of the above step S102, and the above step S104 will be described below.

[0085] In some embodiments of the step S104, the coincidence degree between the second nearest neighbor data corresponding to each of the first profile data can also be quickly and accurately calculated by the aforementioned formula (1), so as to improve the processing efficiency and accuracy of the profile merging. The difference from the step S102 is that, when the coincidence degree between the second nearest neighbor data is calculated by the aforementioned formula (1), the meanings of the parameters in the formula (1) are as follows: N represents a counting function, and the counting function can be used to obtain the number of the second nearest neighbor data corresponding to each of the first profile data in the first to-be-detected profile in the second data library. i (k) the number of the first profile data in the intersection of the first profile data and the n j (k) the number of the first profile data in the intersection of the first profile data and the n ij represents the coincidence degree between the second nearest neighbor data corresponding to the i-th first profile data and the j-th first profile data in the first to-be-detected profile, n i (k) and n j (k) respectively represent the k second nearest neighbor data corresponding to the i-th first profile data and the j-th first profile data in the first to-be-detected profile, k > 1.

[0086] In some other embodiments of the step S104, the second nearest neighbor coincidence degree of the first to-be-detected profile in the second data library can also be accurately determined by the aforementioned formula (2), so as to further improve the accuracy of the profile merging. The difference from the step S102 is that, when the second nearest neighbor coincidence degree is calculated by the aforementioned formula (2), the meanings of the parameters in the formula (2) are as follows: D base represents the second data library, S represents the first to-be-detected profile, sn(S1, D base ) represents the second nearest neighbor coincidence degree of the first to-be-detected profile in the second data library, n represents the total number of the first profile data in the first to-be-detected profile, psn ij represents the coincidence degree between the second nearest neighbor data corresponding to the i-th first profile data and the j-th first profile data in the first to-be-detected profile.

[0087] The above is a further description of the step S104, and the step S105 will be described below.

[0088] Referring to FIG. 1, the step S105 is described as follows. Figure 2 In some embodiments of the step S105, whether the first to-be-detected profile and the second to-be-detected profile are merged can be determined according to the first nearest neighbor coincidence degree and the second nearest neighbor coincidence degree, and by the following steps S1051 to S1054, so as to accurately determine whether the first to-be-detected profile and the second to-be-detected profile are merged, and improve the accuracy of the profile merging.

[0089] Step S1051: Determine the overlap deviation between the nearest neighbor overlap of the second base storage and the nearest neighbor overlap of the first base storage.

[0090] If the nearest neighbor overlap of the second base reservoir is greater than that of the first base reservoir, then the overlap deviation is a number greater than zero; otherwise, the overlap deviation is a number less than zero.

[0091] Step S1052: Based on the comparison result of the overlap deviation and the preset deviation threshold, selectively merge the first file to be detected and the second file to be detected.

[0092] The preset deviation threshold is a value greater than zero.

[0093] If the overlap deviation is greater than or equal to the preset deviation threshold, it indicates that the nearest neighbor overlap of the second database is greater than that of the first database and the difference between the two is large. This means that there are many second files in the second file to be tested that are similar to the first file data in the first file to be tested. Therefore, the first file to be tested and the second file to be tested can be merged. Thus, at this point, we can proceed to step S1053.

[0094] If the overlap deviation is less than the preset deviation threshold, it indicates that the difference between the nearest neighbor overlap of the second database and the nearest neighbor overlap of the first database is small. Therefore, there are not many second files in the second file to be tested that are similar to the first file data in the first file to be tested. Thus, merging the first and second files to be tested is not necessary. Therefore, at this point, the process can proceed to step S1054.

[0095] Step S1053: Merge the first file to be tested and the second file to be tested.

[0096] Step S1054: Do not merge the first file to be tested and the second file to be tested.

[0097] By using the methods described in steps S1051 to S1054 above, it is possible to accurately determine whether to merge the first file to be tested and the second file to be tested, thereby improving the accuracy of file merging.

[0098] Furthermore, in some other embodiments of step S105 above, in order to reduce the impact of interference data (interference data can be file data from which data features cannot be accurately extracted, for example, when the file data is a face image, interference data can be a low-quality face image with low brightness) present in the first file to be detected and the second file to be detected on file merging, step S1053 in the method described in steps S1051 to S1054 above can be replaced with Figure 3 The following steps S201 to S205 are shown.

[0099] Step S201: Forming a third data base according to the second archive data in the second to-be-detected archive and the interference data in the preset interference set.

[0100] It should be noted that the preset interference set in step S201 is completely the same as the preset interference set in the aforementioned step S101 and step S103. Moreover, the interference data in the preset interference set is also the same data type as the third archive data, but does not belong to the same target object.

[0101] In the embodiment, the total number of the second archive data in the second to-be-detected archive is greater than the total number of the first archive data in the first to-be-detected archive. If the first to-be-detected archive and the second to-be-detected archive do not satisfy this condition, it is necessary to modify the first to-be-detected archive into the second to-be-detected archive, modify the second to-be-detected archive into the first to-be-detected archive, i.e., exchange the first to-be-detected archive and the second to-be-detected archive, and then continue to execute the method described in the aforementioned steps S201 to S205.

[0102] Step S202: Retrieving the third base nearest neighbor data of each second archive data in the third data base, and determining the third base nearest neighbor coincidence degree of the second to-be-detected archive in the third data base according to the coincidence degree between the third base nearest neighbor data corresponding to different second archive data.

[0103] The greater the third base nearest neighbor coincidence degree is, the higher the correlation between all the second archive data in the third data base is, and vice versa.

[0104] In the embodiment, the third base nearest neighbor data of each second archive data in the third data base can also be retrieved by using the conventional nearest neighbor algorithm in the data retrieval technical field, for example, the K- nearest neighbor algorithm can still be used for retrieval.

[0105] Further, in some embodiments, in order to improve the retrieval accuracy of the nearest neighbor data, the initial third base nearest neighbor data of the second archive data can be retrieved in the third data base by using the nearest neighbor algorithm, and then the initial third base nearest neighbor data can be filtered according to the similarity between the second archive data and each initial third base nearest neighbor data to obtain the final third base nearest neighbor data. Specifically, the third base nearest neighbor data with a similarity greater than a preset similarity threshold can be selected as the final third base nearest neighbor data. The person skilled in the art can flexibly set the specific value of the preset similarity threshold according to the actual demand, and the embodiment does not make a specific limitation on this.

[0106] Further, in some embodiments of the step S202, the overlap degree between the third data library nearest neighbor data corresponding to different second archive data can also be quickly and accurately calculated by the aforementioned formula (1), so as to improve the processing efficiency and accuracy of the archive merging. The difference from the aforementioned step S102 or step S104 is that, when the overlap degree between the third data library nearest neighbor data is calculated by the aforementioned formula (1), the meanings of the parameters in formula (1) are as follows: N represents a counting function, and the counting function can be used to obtain the number of the second archive data in the intersection of the first data library nearest neighbor data and the third data library nearest neighbor data corresponding to the second archive data in the second to-be-detected archive. i (k) and the n j (k) form the intersection. psn ij represents the overlap degree between the third data library nearest neighbor data corresponding to the i-th second archive data and the j-th second archive data in the second to-be-detected archive, n i (k) and the n j (k) respectively represent the k third data library nearest neighbor data corresponding to the i-th second archive data and the j-th second archive data in the second to-be-detected archive, k > 1.

[0107] Further, in some embodiments of the step S202, the overlap degree between the third data library nearest neighbor data corresponding to different second archive data can also be quickly and accurately calculated by the aforementioned formula (1), so as to improve the processing efficiency and accuracy of the archive merging. The difference from the aforementioned step S102 or step S104 is that, when the overlap degree between the third data library nearest neighbor data is calculated by the aforementioned formula (1), the meanings of the parameters in formula (1) are as follows: N represents a counting function, and the counting function can be used to obtain the number of the second archive data in the intersection of the first data library nearest neighbor data and the third data library nearest neighbor data corresponding to the second archive data in the second to-be-detected archive. base represents the third data library, S represents the second to-be-detected archive, sn(S1, D base ) represents the third data library nearest neighbor overlap degree of the second to-be-detected archive in the third data library, n represents the total number of the second archive data in the second to-be-detected archive, psn ij represents the overlap degree between the third data library nearest neighbor data corresponding to the i-th second archive data and the j-th second archive data in the second to-be-detected archive.

[0108] Step S203: determining whether the overlap degree deviation is greater than or equal to a preset deviation threshold value and whether the first data library nearest neighbor overlap degree and the third data library nearest neighbor overlap degree are greater than or equal to a preset overlap degree threshold value, respectively; if yes, turning to step S204; if no, turning to step S205.

[0109] According to the foregoing step S1052, if the coincidence degree deviation is greater than or equal to the preset deviation threshold, it indicates that the second database nearest neighbor coincidence degree is greater than the first database nearest neighbor coincidence degree and the difference is large, and there are more second archive data similar to the first archive data in the second to-be-detected archive. Therefore, the first to-be-detected archive and the second to-be-detected archive can be merged. However, there may be some interference data in the first to-be-detected archive and the second to-be-detected archive, and if the archives are merged, the accuracy of the merged archives will be reduced. Further, according to the foregoing step S102, the greater the first database nearest neighbor coincidence degree, the higher the correlation between all first archive data in the first database, that is, the less the interference data in the first to-be-detected archive. According to the foregoing step S202, the greater the third database nearest neighbor coincidence degree, the higher the correlation between all second archive data in the third database, that is, the less the interference data in the second to-be-detected archive. Therefore, in this embodiment, the first database nearest neighbor coincidence degree, the third database nearest neighbor coincidence degree, and the preset deviation threshold can be compared. If the first database nearest neighbor coincidence degree is greater than the preset deviation threshold, it indicates that the first database nearest neighbor coincidence degree is large, and the interference data in the first to-be-detected archive is less. If the third database nearest neighbor coincidence degree is greater than the preset deviation threshold, it indicates that the third database nearest neighbor coincidence degree is large, and the interference data in the second to-be-detected archive is less. Under the condition that the coincidence degree deviation is greater than or equal to the preset deviation threshold and the first database nearest neighbor coincidence degree and the third database nearest neighbor coincidence degree are greater than or equal to the preset coincidence degree threshold, the first to-be-detected archive and the second to-be-detected archive are merged, otherwise, the archives are not merged.

[0110] Step S204: merging the first to-be-detected archive and the second to-be-detected archive.

[0111] Step S205: not merging the first to-be-detected archive and the second to-be-detected archive.

[0112] Through the method described in the foregoing steps S201 to S205, the interference data in the first to-be-detected archive and the second to-be-detected archive can be reduced, the influence of the archive merging is reduced, and the accuracy of the archive merging is further improved.

[0113] It should be noted that although the above embodiments describe the steps in a specific order, those skilled in the art can understand that, in order to achieve the effect of the present application, the different steps do not have to be executed in this order, they can be executed simultaneously (in parallel) or in other orders, and these changes are within the protection scope of the present application.

[0114] Further, the present application also provides an archive merging device.

[0115] Referring to the drawings Figure 4 , Figure 4 is a main structural block diagram of the profile merging device according to an embodiment of the present application. As shown in Figure 4 , the profile merging device in the embodiment of the present application mainly comprises a first database nearest neighbor coincidence degree determining module, a second database nearest neighbor coincidence degree determining module and a profile merging module. In some embodiments, one or more of the first database nearest neighbor coincidence degree determining module, the second database nearest neighbor coincidence degree determining module and the profile merging module can be combined together into one module. In some embodiments, the first database nearest neighbor coincidence degree determining module can be configured to form a first data database according to first profile data in a first to-be-detected profile and interference data in a preset interference set, retrieve first database nearest neighbor data of each first profile data in the first data database, and determine a first database nearest neighbor coincidence degree of the first to-be-detected profile in the first data database according to coincidence degrees between respective corresponding first database nearest neighbor data of different first profile data; the second database nearest neighbor coincidence degree determining module can be configured to form a second data database according to the first profile data in the first to-be-detected profile, second profile data in a second to-be-detected profile and the interference data in the preset interference set, retrieve second database nearest neighbor data of each first profile data in the second data database, and determine a second database nearest neighbor coincidence degree of the first to-be-detected profile in the second data database according to coincidence degrees between respective corresponding second database nearest neighbor data of different first profile data; and the profile merging module can be configured to determine whether to perform profile merging on the first to-be-detected profile and the second to-be-detected profile according to the first database nearest neighbor coincidence degree and the second database nearest neighbor coincidence degree. In one implementation, the description of the specific functions can be referred to the description of steps S101 to S105.

[0116] The profile merging device described above is used to execute the profile merging method embodiment shown in Figures 1 to 3 , and the technical principles, the technical problems solved and the technical effects generated are similar. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the profile merging device can be referred to the description of the profile merging method embodiment, which will not be described here.

[0117] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable storage medium can include any entity or device, medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable storage medium does not include electrical carrier signals and telecommunication signals.

[0118] Further, the present application also provides a computer device. In an embodiment of the computer device according to the present application, the computer device comprises a processor and a storage device, the storage device can be configured to store a program for executing the archive merging method of the above-mentioned method embodiments, and the processor can be configured to execute the program in the storage device, which includes but is not limited to the program for executing the archive merging method of the above-mentioned method embodiments. For the convenience of illustration, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The computer device can be a control device formed by various electronic devices.

[0119] Further, the present application also provides a computer readable storage medium. In an embodiment of the computer readable storage medium according to the present application, the computer readable storage medium can be configured to store a program for executing the archive merging method of the above-mentioned method embodiments, which can be loaded and run by the processor to implement the above-mentioned archive merging method. For the convenience of illustration, only the parts related to the embodiments of the present application are shown, and the specific technical details not disclosed are referred to the method part of the embodiments of the present application. The computer readable storage medium can be a storage device formed by various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.

[0120] Further, it should be understood that, since the setting of each module is only for illustrating the functional units of the device of the present application, the physical device corresponding to the module can be the processor itself, or a part of software, a part of hardware or a part of combination of software and hardware in the processor. Therefore, the number of each module in the figure is only illustrative.

[0121] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solution to deviate from the principles of the present application, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present application.

[0122] So far, the technical solution of the present application has been described in combination with one embodiment shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without deviating from the principles of the present application, and the technical solutions after the changes or replacements will fall within the protection scope of the present application.

Claims

1. An archival merge method, characterized by, The method comprises: forming a first data base according to first archive data in a first to-be-detected archive and interference data in a preset interference set, searching for first base nearest neighbor data of each of the first archive data in the first data base, and determining a first base nearest neighbor coincidence degree of the first to-be-detected archive in the first data base according to coincidence degrees between the first base nearest neighbor data corresponding to different first archive data respectively; forming a second data base according to the first archive data in the first to-be-detected archive, second archive data in a second to-be-detected archive and the interference data in the preset interference set, searching for second base nearest neighbor data of each of the first archive data in the second data base, and determining a second base nearest neighbor coincidence degree of the first to-be-detected archive in the second data base according to coincidence degrees between the second base nearest neighbor data corresponding to different first archive data respectively; determining whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first base nearest neighbor coincidence degree and the second base nearest neighbor coincidence degree; wherein the determining whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive comprises: determining a coincidence degree deviation between the second base nearest neighbor coincidence degree and the first base nearest neighbor coincidence degree; selectively performing archive merging on the first to-be-detected archive and the second to-be-detected archive according to a comparison result of the coincidence degree deviation and a preset deviation threshold; if the coincidence degree deviation is greater than or equal to the preset deviation threshold, performing archive merging; otherwise, not performing archive merging.

2. The archive merge method of claim 1, wherein, The step of "selectively performing archive merging on the first to-be-detected archive and the second to-be-detected archive according to a comparison result of the coincidence degree deviation and a preset deviation threshold" further comprises: forming a third data base according to the second archive data in the second to-be-detected archive and the interference data in the preset interference set, searching for third base nearest neighbor data of each of the second archive data in the third data base, and determining a third base nearest neighbor coincidence degree of the second to-be-detected archive in the third data base according to coincidence degrees between the third base nearest neighbor data corresponding to different second archive data respectively; determining whether the coincidence degree deviation is greater than or equal to the preset deviation threshold and whether the first base nearest neighbor coincidence degree and the third base nearest neighbor coincidence degree are greater than or equal to a preset coincidence degree threshold respectively; if yes, performing archive merging; if no, not performing archive merging.

3. The archive merge method of claim 2, wherein, The method further comprises determining the first base nearest neighbor coincidence degree or the second base nearest neighbor coincidence degree or the third base nearest neighbor coincidence degree respectively by the following way: determining the first base nearest neighbor coincidence degree or the second base nearest neighbor coincidence degree or the third base nearest neighbor coincidence degree respectively according to coincidence degrees between the first base nearest neighbor data or the second base nearest neighbor data or the third base nearest neighbor data and by the following formula: wherein, When D base S represents the first to-be-detected file when representing the first data library, sn(S1, D base ) represents the first library nearest neighbor degree of the first to-be-detected file, n represents the total number of first file data in the first to-be-detected file, psn ij represents the coincidence degree between the first library nearest neighbor data corresponding to the i-th first file data and the j-th first file data in the first to-be-detected file; When D base S represents the first file to be detected, sn(S1, D base ) represents the second database nearest neighbor coincidence degree, n represents the total number of first file data in the first file to be detected, psn ij represents the coincidence degree between the second database nearest neighbor data corresponding to the i-th first file data and the j-th first file data in the first file to be detected; When D base S represents the second to-be-detected file when the third data library is represented, sn(S1, D base ) represents the third library nearest neighbor coincidence degree, n represents the total number of second file data in the second to-be-detected file, psn ij represents the coincidence degree between the third library nearest neighbor data corresponding to the i-th second file data and the j-th second file data in the second to-be-detected file.

4. The archive merge method of claim 2, wherein, The method further comprises calculating the coincidence degree between the first database nearest neighbor data or the second database nearest neighbor data or the third database nearest neighbor data by the following formula respectively: psn ij = N(n i (k)∩n j (k)) Wherein, N represents the counting function; when psn ij n represents the coincidence degree between the i-th first archive data and the j-th first archive data in the first to-be-detected archive and the first database nearest neighbor data corresponding to the i-th first archive data and the j-th first archive data. i (k) and n j (k) respectively represent k first database nearest neighbor data corresponding to the i-th first archive data and the j-th first archive data in the first to-be-detected archive, k>1. when psn ij n represents the degree of coincidence between the i-th first archive data and the j-th first archive data in the first archive to be detected and the second database nearest neighbor data corresponding to the i-th first archive data and the j-th first archive data. i (k) and n j (k) respectively represent the k second database nearest neighbor data corresponding to the i-th first archive data and the j-th first archive data in the first archive to be detected, k > 1. when psn ij n represents the degree of coincidence between the third database nearest neighbor data corresponding to the i-th second archive data and the j-th second archive data in the second archive to be detected. i (k) and n j (k) represent the k third database nearest neighbor data corresponding to the i-th second archive data and the j-th second archive data in the second archive to be detected, respectively, k > 1.

5. The archive merge method of claim 2, wherein, The method further comprises retrieving the first database nearest neighbor data or the second database nearest neighbor data or the third database nearest neighbor data by the following ways respectively: Using the nearest neighbor algorithm, the initial database nearest neighbor data of the to-be-retrieved archive data is retrieved in the to-be-retrieved database; According to the similarity between the to-be-retrieved archive data and each of the initial database nearest neighbor data, the initial database nearest neighbor data is filtered to obtain the final database nearest neighbor data; Wherein, When the to-be-retrieved database is the first data database and the to-be-retrieved archive data is the first archive data, the initial database nearest neighbor data is the initial first database nearest neighbor data, and the final database nearest neighbor data is the final first database nearest neighbor data. When the to-be-retrieved database is the second data database and the to-be-retrieved archive data is the first archive data, the initial database nearest neighbor data is the initial second database nearest neighbor data, and the final database nearest neighbor data is the final second database nearest neighbor data. When the to-be-retrieved database is the third data database and the to-be-retrieved archive data is the second archive data, the initial database nearest neighbor data is the initial third database nearest neighbor data, and the final database nearest neighbor data is the final third database nearest neighbor data.

6. The archive merge method of claim 2, wherein, The total number of the second archive data in the second to-be-detected archive is greater than the total number of the first archive data in the first to-be-detected archive.

7. An archival merge apparatus, comprising: The device comprises: A first database nearest neighbor coincidence degree determination module configured to form a first data database according to the first archive data in the first to-be-detected archive and the interference data in a preset interference set, retrieve the first database nearest neighbor data of each of the first archive data in the first data database, and determine the first database nearest neighbor coincidence degree of the first to-be-detected archive in the first data database according to the coincidence degree between the first database nearest neighbor data corresponding to different first archive data respectively; A second database nearest neighbor coincidence degree determination module configured to form a second data database according to the first archive data in the first to-be-detected archive, the second archive data in the second to-be-detected archive, and the interference data in the preset interference set, retrieve the second database nearest neighbor data of each of the first archive data in the second data database, and determine the second database nearest neighbor coincidence degree of the first to-be-detected archive in the second data database according to the coincidence degree between the second database nearest neighbor data corresponding to different first archive data respectively; An archive merging module configured to determine whether to perform archive merging on the first to-be-detected archive and the second to-be-detected archive according to the first database nearest neighbor coincidence degree and the second database nearest neighbor coincidence degree. The determination of whether to perform the archive merging on the first to-be-detected archive and the second to-be-detected archive comprises: determining a coincidence degree deviation between the second database nearest neighbor coincidence degree and the first database nearest neighbor coincidence degree; selectively performing the archive merging on the first to-be-detected archive and the second to-be-detected archive according to a comparison result of the coincidence degree deviation and a preset deviation threshold; and performing the archive merging if the coincidence degree deviation is greater than or equal to the preset deviation threshold; otherwise, not performing the archive merging.

8. A computer device comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the archive merging method of any one of claims 1 to 6.

9. A computer readable storage medium having stored therein a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the archive merging method of any one of claims 1 to 6.

Images