A method and system for processing a large amount of real-time features based on documents

Abstract

The present application relates to the field of electronic digital data processing technology, and in particular to a method and system for massive real-time feature processing based on documents. The method comprises the following steps: according to K videos acquired within a target time period, doc=(doc1, doc2, ..., doc K ); traverse doc, if doc i,meta,1 The unique ID of the camera that the user is searching for, and doc i,meta,2 If the time range overlaps with the time range searched by the user, the doc is obtained. i,data,2,j docs within the time range searched by the user i,data,3,j ; Get Sim j,0 The largest first N docs i,data,1,j ; Sim j,0 The largest first N docs i,data,1,j Aggregation is performed to obtain the ID of the ReID feature searched by the user. The present invention realizes rapid search of massive real-time features.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a method and system for processing a large amount of real-time features based on documents. Background Art

[0002] Person Re-identification (ReID) is a technology that uses computer vision technology to determine whether a specific pedestrian exists in an image. The ReID features based on images can be used to query images with specific pedestrians from a large number of images. Traditional large-scale real-time feature search systems generally store ReID features in a key-value database and store ReID feature metadata in a relational database; during search, all keys of ReID features that meet the search conditions are first filtered out from the relational database according to the search conditions, and then the ReID feature values corresponding to the keys are read from the key-value database to establish a temporary search index, and the search results are returned. However, the upper limit of the queries per second (QPS) of the key-value database is too low. Even for an in-memory key-value database such as Redis, the single-machine QPS is less than 100,000. Therefore, the speed of reading features from the key-value database is relatively slow, which affects the speed of feature processing. How to achieve fast search of a large amount of real-time features is a technical problem to be solved urgently. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for processing a large amount of real-time features based on documents to achieve fast search of a large amount of real-time features.

[0004] According to the first aspect of the present invention, a method for processing a large amount of real-time features based on documents is provided, including the following steps:

[0005] S100, according to K videos obtained within a target time period, obtain doc=(doc1, doc2,..., doc K ), doc i is the document information corresponding to P i , P i is the i-th video obtained within the target time period, the value range of i is from 1 to K, K is the total number of videos obtained within the target time period, doc i ={doc i,meta , doc i,data}, doc i,meta is the document metadata in doc i , doc i,meta includes doc i,meta,1 and doc i,meta,2 , doc i,meta,1 is Pi The unique identifier of the affiliated camera, doc i,meta,2 is P i The corresponding time range; doc i,data is doc i The document data in doc i,data ={doc i,data,1 , doc i,data,2 , doc i,data,3}, doc i,data,1 is the ID of the ReID feature of P i =(doc i,data,1 , doc i,data,1,1 , doc i,data,1,2 , …, doc i,data,1,Mi ), doc i,data,1,j is the ID of the ReID feature of P i,j The ID of the ReID feature of P, P i,j is P i The j-th frame video image in P, where the value range of j is from 1 to Mi, and Mi is the total number of video images in P i The total number of video images in doc i,data,2 is the timestamp of the ReID feature of P i =(doc i,data,2 , doc i,data,2,1 , doc i,data,2,2 , …, doc i,data,2,Mi ), doc i,data,2,j is the timestamp of the ReID feature of P i,j The timestamp of the ReID feature of P, doc i,data,3 is P i The ReID feature of P i,data,3 =(doc i,data,3,1 , doc i,data,3,2 , …, doc i,data,3,Mi ), doc i,data,3,j is P i,j The ReID feature of P

[0006] S200, traverse doc, if doc i,meta,1 is the unique identifier of the camera searched by the user, and doc i,meta,2 is the time range searched by the user, then obtain doc i,data,2,j within the time range searched by the user i,data,3,j .

[0007] S300, traverse doc, obtain the top N docs with the largest Sim j,0 , where Sim i,data,1,j is doc j,0 within the time range searched by the user i,data,2,j and doc i,data,3,jThe similarity to the ReID features searched by the user, where N is a preset quantity.

[0008] S400, traverse the docs, and use Sim j,0 The top N docs with the largest i,data,1,j Aggregate them to obtain the IDs of the ReID features searched by the user.

[0009] Compared with the prior art, the present invention has obvious beneficial effects. By means of the above technical solution, a method for processing massive real-time features based on documents provided by the present invention can achieve considerable technical progressiveness and practicality, and has wide utilization value in the industry. It has at least the following beneficial effects:

[0010] The present invention stores the ReID features of a video in a document data. Each document data includes the ReID features of multiple consecutive frames of images. The document metadata corresponding to each document data includes the unique identifier of the camera to which the corresponding video belongs and the corresponding time range. When the unique identifier of the camera and the time range included in a certain document metadata are not the unique identifier of the camera and the time range searched by the user, it can be directly determined that all the ReID features included in the corresponding document data are not the ReID features searched by the user. Thus, the present invention can reduce the number of times of determining whether each ReID feature in the document data is the ReID feature searched by the user, and improve the search speed of massive real-time features.

[0011] The present invention stores ReID features in a document manner, which can maximize the continuous reading of ReID features. Compared with the existing method of storing using a Key-Value database, the QPS can easily reach the million level, realizing the fast search of massive real-time features. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0013] Figure 1 It is a flowchart of the method for processing massive real-time features based on documents provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0015] According to a first aspect of the present invention, there is provided a method for processing a large amount of real-time features based on documents, as Figure 1 shown, including the following steps:

[0016] S100, according to K videos obtained within a target time period, obtain doc = (doc1, doc2,..., doc K ), doc i is the document information corresponding to P i , P i is the i-th video obtained within the target time period, the value range of i is from 1 to K, K is the total number of videos obtained in the target time period, doc i ={doc i,meta , doc i,data}, doc i,meta is the document metadata in doc i , doc i,meta includes doc i,meta,1 and doc i,meta,2 , doc i,meta,1 is the unique identifier of the camera to which P i belongs, doc i,meta,2 is the time range corresponding to P i ; doc i,data is the document data in doc i , doc i,data ={doc i,data,1 , doc i,data,2 , doc i,data,3}, doc i,data,1 is the ID of the ReID feature of P i , doc i,data,1 =(doc i,data,1,1 , doc i,data,1,2 ,..., doc i,data,1,Mi ), doc i,data,1,j is the ID of the ReID feature of P i,j , P i,j is the j-th video image in P i , the value range of j is from 1 to Mi, Mi is the total number of video images in P i , doc i,data,2 is P iTimestamp of the ReID feature of, doc i,data,2 =(doc i,data,2,1 , doc i,data,2,2 , …, doc i,data,2,Mi ), doc i,data,2,j is the timestamp of the ReID feature of P i,j Timestamp of the ReID feature of, doc i,data,3 is the ReID feature of P i i,data,3 =(doc i,data,3,1 , doc, doc i,data,3,2 , …, doc i,data,3,Mi ), doc i,data,3,j is the ReID feature of P i,j .

[0017] It should be understood that K is a positive integer and Mi is a positive integer. There are differences in the unique identifiers of the cameras to which the K videos obtained within the target time period belong and the corresponding time ranges. The time range corresponding to the video refers to the shooting time corresponding to the video. For example, the unique identifiers of the cameras to which P1 and P6 belong are the same, but the time ranges corresponding to P1 and P6 are different. The time range corresponding to P1 is 1:00 - 2:00, and the time range corresponding to P6 is 3:00 - 3:30; the time ranges corresponding to P2 and P3 are the same, but the unique identifiers of the cameras to which they belong are different. The unique identifier of the camera to which P2 belongs is 1, and the unique identifier of the camera to which P3 belongs is 4; the unique identifiers of the cameras to which P4 and P5 belong and the corresponding time ranges are all different.

[0018] Preferably, store doc i,data in binary. Storing doc i,data in binary has the advantages of less occupied space and can improve the search speed.

[0019] Preferably, use SQLite to store doc i,meta . Using SQLite to store doc i,meta has the advantages of simple and efficient storage.

[0020] Preferably, use a dix file to store doc i,data,1 , use a ts file to store doc i,data,2 , use a dat file to store doc i,data,3 . Storing using the three files has the advantages of simple storage, convenient positioning, and relatively high storage efficiency.

[0021] Optionally, doc i,data,1 is of numeric type or string type, doc i,data,2 is of int64 type, doc i,data,3It is of the Float array type. Further, the numerical type is int64 type.

[0022] It can be understood that the ReID feature refers to a one-dimensional Float array extracted from a pedestrian picture. As a specific implementation, the ReID feature is a one-dimensional Float array with a dimension of 512. Then, the dix file will write 8 bytes to store the doc i,data,1,j , and the ts file will write 8 bytes to store the doc i,data,2,j , and the dat file will write 512 * 4 = 2048 bytes to store the doc i,data,2,j .

[0023] S200, traverse the doc. If the doc i,meta,1 is the unique identifier of the camera searched by the user, and the doc i,meta,2 has an intersection with the time range searched by the user, then obtain the doc i,data,2,j within the time range searched by the user i,data,3,j .

[0024] It should be understood that the doc i,meta,2 having an intersection with the time range searched by the user includes the following situations: the doc i,meta,2 is equal to the time range searched by the user, and only some of the doc i,data,2 belong to the time range searched by the user. Only the doc i,data,2,j having an intersection with the time range searched by the user i,meta,2 is the document data to be searched subsequently. The starting and ending offsets of the doc i,data,2 to be searched can be determined according to the time range searched by the user, and then the doc i,data,2 within the time range searched by the user i,data,2,j can be determined i,data,3,j .

[0025] As a specific implementation, the time range searched by the user is 3:00 - 4:00. If the doc i,meta,2 is 3:00 - 4:00, then the doc i,meta,2 is equal to the time range searched by the user. The starting offset of the doc i,data,2 to be searched indicates the doc i,data,2,j at 3:00, and the ending offset indicates the doc i,data,3,j at 4:00 i,data,2,j ; if the doc i,data,3,j is 3:30 - 4:30, then only some of the doc i,meta,2 belong to the time range searched by the user, and the doc i,data,2 among them i,data,2,j belong to the time range searched by the useri,data,2 The starting offset to be searched indicates the doc i,data,2,j for the doc at 3:30 i,data,3,j , and the ending offset indicates the doc i,data,2,j for the doc at 4:00 i,data,3,j ; if the doc i,meta,2 is from 4:30 to 5:00, then the doc i,meta,2 has no intersection with the time range of the user's search, and the doc i,data,2 is not the document data to be searched.

[0026] Preferably, the doc i,meta also includes the doc i,meta,3 and the doc i,meta,4 , the doc i,meta,3 is the doc i the number of ReID features included, the doc i,meta,4 is the doc i whether it is the tag being written. If Mi = doc i,meta,3 , then the doc i,meta,4 indicates that the doc i has been written, and it is necessary to create the doc i+1 to continue writing; if Mi < doc i,meta,3 , then the doc i,meta,4 indicates that the doc i is being written, and writing can continue in the doc i .

[0027] Preferably, obtain the doc i,data,2,j within the time range of the user's search, the doc i,data,3,j includes:

[0028] S210, if the doc i,meta,4 indicates that the doc i has been written, then execute S221; if the doc i,meta,4 indicates that the doc i is being written, then execute S222.

[0029] S221, if the doc i,data,2 is stored in the memory cache of the written document data, then obtain the doc i,data,2,j within the time range of the user's search from the memory cache of the written document data i,data,3,j ; if the doc i,data,2 is not stored in the memory cache of the written document data, then obtain the doc i,data,2,j within the time range of the user's search from the updated memory cache of the written document data i,data,3,j; The updated in-memory cache of the written document data is the in-memory cache of the written document data after storing doc i,data,2 in the in-memory cache of the written document data after storing doc

[0030] S222, if doc i,data,2 is stored in the in-memory cache of the document data being written, then obtain doc i,data,2,j within the time range of the user's search from the in-memory cache of the document data being written i,data,3,j ; If doc i,data,2 is not stored in the in-memory cache of the document data being written, then obtain doc i,data,2,j within the time range of the user's search from the updated in-memory cache of the document data being written i,data,3,j ; The updated in-memory cache of the document data being written is the in-memory cache of the document data being written after storing doc i,data,2 in the in-memory cache of the document data being written after storing doc

[0031] For the document data being written, it is necessary to continuously check whether the document data has been updated to determine whether to update the in-memory cache of the document data being written, which involves the problem of multi-threaded in-memory cache update of document data; while for the written document data, there is no need to continuously check whether the document data has been updated, so there is no need to update the written document data in the in-memory cache of the written document data, and its efficiency is higher than that of the in-memory cache of the document data being written

[0032] S221 and S222 obtain doc i,data,2,j within the time range of the user's search from the in-memory cache or the updated in-memory cache i,data,3,j , compared with obtaining doc i,data,2,j within the time range of the user's search from the hard disk i,data,3,j , since doc i,data,2 is read from the in-memory cache, the reading speed is relatively fast, which can improve the search speed for features

[0033] Preferably, traversing doc includes:

[0034] S201, divide doc to obtain Z document information blocks, each document information block includes at least 1 piece of document information corresponding to a video, and Z≥2

[0035] S202, traverse doc by traversing the Z document information blocks simultaneously

[0036] Adopting the above method of traversing the document information blocks simultaneously can reduce the traversal time and improve the search speed for features. It should be understood that Z is a positive integer

[0037] As a specific implementation, according to doc i,meta,1 Divide doc, doc i,meta,1 is 1 - 18. After dividing doc, 3 document information blocks can be obtained. The first document information block corresponds to doc i,meta,1 with doc being 1 - 6 i , the second document information block corresponds to doc i,meta,1 with doc being 7 - 12 i , and the third document information block corresponds to doc i,meta,1 with doc being 13 - 18 i . When traversing doc, the first, second, and third document information blocks can be traversed simultaneously, improving the traversal speed and saving traversal time.

[0038] S300, traverse doc to obtain the top N doc with the largest Sim j,0 , where Sim i,data,1,j is the similarity between doc j,0 within the time range of the user's search and the ReID feature of the user's search, and N is a preset quantity. i,data,2,j within the time range of the user's search i,data,3,j and the ReID feature of the user's search are both vectors. Those skilled in the art know that any method for calculating the similarity of vectors in the prior art can be used to calculate Sim

[0039] It should be understood that N is a positive integer. Doc i,data,2,j within the time range of the user's search i,data,3,j and the ReID feature of the user's search are both vectors. Those skilled in the art know that any method for calculating the similarity of vectors in the prior art can be used to calculate Sim j,0 all fall within the protection scope of the present invention.

[0040] Preferably, use the above - mentioned method of traversing Z document information blocks simultaneously to traverse doc, so as to reduce the traversal time and improve the search speed for features.

[0041] S400, traverse doc and aggregate the top N doc with the largest Sim j,0 to obtain the ID of the ReID feature searched by the user. i,data,1,j

[0042] Preferably, use the above - mentioned method of traversing Z document information blocks simultaneously to traverse doc, so as to reduce the traversal time and improve the search speed for features.

[0043] ​Thus, after the user inputs the time range to be searched, the unique identifier of the camera to be searched, and the image ReID feature to be searched, the present invention can return to the user the IDs of multiple ReID features that meet the user's search time range and the unique identifier of the camera and are most similar to the image ReID feature to be searched. Since the ID of the ReID feature of each frame of video image is unique, the user can view the corresponding most similar multiple frames of video images according to the IDs of the multiple most similar ReID features obtained.

[0044] An embodiment of the present invention further provides a massive real-time feature processing system based on documents, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the above method is implemented.

[0045] Although some specific embodiments of the present invention have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A method for processing a large amount of real-time features based on documents, characterized in that, Including: S100. Based on the K videos obtained within the target time period, obtain doc=(doc1, doc2, …, doc K ), where doc i is the document information corresponding to P i , P i is the i-th video obtained within the target time period, where the value range of i is from 1 to K, and K is the total number of videos obtained within the target time period. doc i ={doc i,meta , doc i,data}, where doc i,meta is the document metadata in doc i . doc i,meta includes doc i,meta,1 and doc i,meta,2 . doc i,meta,1 is the unique identifier of the camera to which P i belongs, and doc i,meta,2 is the time range corresponding to P i . doc i,data is the document data in doc i . doc i,data ={doc i,data,1 , doc i,data,2 , doc i,data,3}, where doc i,data,1 is the ID of the ReID feature of P i . doc i,data,1 =(doc i,data,1,1 , doc i,data,1,2 , …, doc i,data,1,Mi ), where doc i,data,1,j is the ID of the ReID feature of P i,j . P i,j is the j-th video image in P i , where the value range of j is from 1 to Mi, and Mi is the total number of video images in P i . doc i,data,2 is the timestamp of the ReID feature of P i . doc i,data,2 =(doc i,data,2,1 , doc i,data,2,2 , …, doc i,data,2,Mi ), where doc i,data,2,j is the timestamp of the ReID feature of P i,j . doc i,data,3 is the ReID feature of P i . doc i,data,3 =(doc i,data,3,1 , doc i,data,3,2 , …, doc i,data,3,Mi ), doc i,data,3,j is the ReID feature of P i,j ; S2 00, traverse the doc, if the doc i,meta,1 is the unique identifier of the camera searched by the user, and the doc i,meta,2 has an intersection with the time range searched by the user, then obtain the doc i,data,2,j within the time range searched by the user i,data,3,j ; S300 traverses the doc to obtain Sim j,0 The top N largest docs i,data,1,j , Sim j,0 is the doc i,data,2,j within the time range of the user's search i,data,3,j and the similarity to the ReID feature searched by the user, where N is a preset quantity; S400 traverses the doc and gets Sim j,0 The top N largest docs i,data,1,j Perform aggregation to obtain the ID of the ReID feature searched by the user.

2. The method according to claim 1, wherein In S100, doc i,meta also includes doc i,meta,3 and doc i,meta,4 , doc i,meta,3 is doc i the number of ReID features included, doc i,meta,4 is doc i the tag being written. If Mi = doc i,meta,3 , then doc i,meta,4 indicates doc i writing completed; if Mi < doc i,meta,3 , then doc i,meta,4 indicates doc i writing in progress.

3. The method according to claim 2, wherein In S200, obtain the doc i,data,2,j The doc within the time range of the user's search i,data,3,j including: S210, if doc i,meta,4 indicates that doc i has been written, then execute S221; if doc i,meta,4 indicates that doc i is being written, then execute S222; S221, if doc i,data,2 is stored in the in-memory cache of the document data that has been written, then obtain doc from the in-memory cache of the document data that has been written i,data,2,j doc within the time range of the user's search i,data,3,j ; if doc i,data,2 is not stored in the in-memory cache of the document data that has been written, then obtain doc from the updated in-memory cache of the document data that has been written i,data,2,j doc within the time range of the user's search i,data,3,j ; the updated in-memory cache of the document data that has been written is the in-memory cache of the document data that has been written after storing doc i,data,2 ; S222, if doc i,data,2 is stored in the memory cache of the document data being written, then obtain doc from the memory cache of the document data being written i,data,2,j that is within the time range of the user's search i,data,3,j ; if doc i,data,2 is not stored in the memory cache of the document data being written, then obtain doc from the updated memory cache of the document data being written i,data,2,j that is within the time range of the user's search i,data,3,j ; the updated memory cache of the document data being written is the memory cache of the document data being written after storing doc i,data,2 .

4. The method according to claim 1, wherein In S100, store doc in binary form i,data .

5. The method according to claim 1, characterized in that In S100, SQLite is used to store the doc i,meta .

6. The method according to claim 1, wherein In S100, use the dix file to store the doc i,data,1 , use the ts file to store the doc i,data,2 , use the dat file to store the doc i,data,3 .

7. The method according to claim 1, wherein In S100, doc i,data,1 is of int64 type, doc i,data,2 is of int64 type, doc i,data,3 is of Float array type.

8. The method according to claim 1, characterized in that, In S200, traversing the doc includes: S201, dividing the doc to obtain Z document information blocks, each document information block including the document information corresponding to at least 1 video, Z≥2; S202, traversing the doc by a way of traversing the Z document information blocks simultaneously.

9. The method according to claim 8, wherein In S300 and S400, traversing the doc by a way of traversing the Z document information blocks simultaneously.

10. A document-based massive real-time feature processing system, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 9 above is implemented.

Images