Method for identifying, comparing, merging and counting passenger flow in ReID technology

Abstract

The invention discloses a method for identifying, comparing, merging and counting passenger flow numbers in a ReID technology, and relates to the technical field of target detection. The technical key point is that the method is used for matching store entering and leaving data and merging the store entering and leaving data, so that the accuracy of the matching between the store entering-leaving data and the store entering-store entering data is improved. Multi-dimensional parameters are introduced, independent and flexible configuration is carried out for enterprises and stores of different industries, and the matching method is beneficial for improving the accuracy degree of matching and merging between store entering-store leaving data and store entering-store entering data.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and specifically to a method for identifying, comparing, merging and counting passenger flow numbers using ReID technology. Background Art

[0002] As the economy gradually recovers, offline stores are experiencing rapid development, so the demand for accurate statistics of customers visiting stores is also increasing. Therefore, it is necessary to count the number and flow of customers.

[0003] The infrared sensing method is less expensive, but it may result in duplicate counting (for example, customers who forget to buy other things may repeatedly enter and exit offline stores, etc.), resulting in inaccurate customer traffic. When using facial recognition, customers must face the camera and take off their hats and masks, and there must be no uneven lighting or side face obstructions, so that the customer traffic can be accurately counted. However, in practice, this is difficult to do, resulting in inaccurate customer traffic. Therefore, the accuracy of customer traffic counting using infrared sensing and facial recognition is not high. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for identifying, comparing, and merging passenger flow statistics using ReID technology, so as to improve the inaccuracies of the existing matching and merging.

[0005] The above object of the present invention is achieved through the following technical solutions:

[0006] A method for identifying, comparing, and merging passenger flow statistics using ReID technology includes the following steps:

[0007] A: Roughly count the data of passing through / entering the store through the device, including the algorithm detecting the human figure and tracking the trajectory, and uploading the human figure trajectory status after the trajectory is screened to the cloud;

[0008] B: The customer data entering the store is filtered in a refined manner through the cloud, and the feature values ​​of the human figure are extracted through ReID. The data is filtered, screened, and deduplicated, and finally matched and merged to obtain the results.

[0009] In a preferred example, the present invention can be further configured as follows: Humanoid status judgment: establishing judgment rules for passing through a store, entering a store, and leaving a store on a device to judge the event status of each humanoid.

[0010] In a preferred example, the present invention can be further configured as follows: the human figure uploaded to the platform by the device is sent to the algorithm model analysis server, and the personnel are screened using the image recognition algorithm model.

[0011] In a preferred example, the present invention can be further configured as follows: ReID technology is used to extract the feature value of the human figure uploaded to the platform. If multiple human figure images are uploaded, the feature values ​​from the multiple human figure images are weighted and fused into a set of feature values.

[0012] In a preferred example, the present invention can be further configured as follows: the user sets the rules for the clerk's judgment method:

[0013] Compare each store entry / exit human figure with the data in the human figure database and compare the ReID feature value similarity:

[0014] Those who exceed the threshold are judged as store employees and no subsequent comparison or merging is performed;

[0015] Subsequent comparison and merging process will be carried out for those who are determined to be non-store employees.

[0016] In a preferred example, the present invention can be further configured as follows: the comparison includes "in-out" matching and "in-in" merging.

[0017] In a preferred example, the present invention can be further configured as follows: when the user sets the store clerk judgment method to be based on the number of store visits / stay duration, the store clerk data is filtered according to the store clerk judgment rules set by the user.

[0018] In a preferred example, the present invention can be further configured as follows: the data of customers entering the store is processed according to the customer deduplication time period set by the user, and only one valid customer data is recorded if the customer enters the store multiple times within the deduplication time period.

[0019] In summary, the present invention includes at least one of the following beneficial technical effects:

[0020] 1. This method combines human detection and tracking technology, and can improve the accuracy of target detection through multiple filtering and judgment;

[0021] 2. It is not based on a single feature or judgment condition, but takes into account the time when the human figure appears in the camera, whether the store leaving event occurs after the store entering event, the order of events, similarity, and other factors;

[0022] 3. This method provides a variety of conditions and threshold settings at each stage; it can also be applied to different matching market demands that may exist in different industry scenarios to ensure a high accuracy rate;

[0023] The passenger flow counting method based on ReID technology has the advantages of high accuracy, reliability, and flexibility, and can count the actual passenger flow in a variety of scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the device side in this technical solution;

[0025] Figure 2 It is a schematic diagram of the cloud process in this technical solution;

[0026] Figure 3 It is a matrix table in this technical solution;

[0027] Figure 4 It is the matching result table in this technical solution;

[0028] Figure 5 It is the merged table in this technical solution;

[0029] Figure 6 It is the merge result table in this technical solution. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0031] like Figure 1-2 As shown, this is a method disclosed in the technical solution for identifying, comparing, and merging passenger flow statistics using ReID technology:

[0032] A: Roughly count the data of passing through / entering the store through the device, including the algorithm detecting the human figure and tracking the trajectory, and uploading the human figure trajectory status after the trajectory is screened to the cloud;

[0033] Humanoid status judgment: Establish judgment rules for passing through the store, entering the store, and leaving the store on the device to judge the event status of each humanoid. When the humanoid status is passing through the store, upload [Humanoid status = passing through the store] data to the platform. The content uploaded to the platform here includes the uploaded human trajectory status;

[0034] When the humanoid status is entering the store, upload [Humanoid status = entering the store] data and one or more humanoid images to the platform. The content uploaded to the platform here includes: humanoid trajectory status, trajectory start and end time, and X humanoid images with the highest quality;

[0035] When the humanoid status is leaving the store, upload [Humanoid Status = Leaving the Store] data and one or more humanoid images to the platform. The content uploaded to the platform here includes: humanoid trajectory status, trajectory start and end time, and X humanoid images with the highest quality.

[0036] The human figure uploaded from the device to the platform is sent to the algorithm model analysis server, and the image recognition algorithm model for deliverymen / couriers is used to filter the couriers / deliverymen. Filtering is done through the work clothes model, such as Meituan clothing, JD clothing and other models.

[0037] B: The customer data entering the store is filtered in a refined manner through the cloud, and the feature values ​​of the human figure are extracted through ReID. The data is filtered, screened, and deduplicated, and finally matched and merged to obtain the results.

[0038] Specifically: ReID technology is used to extract the feature values ​​of the human figure uploaded to the platform by the device, including age and gender information. If multiple human figure images are uploaded, the feature values ​​from the multiple human figure images will be weighted and fused into a set of feature values.

[0039] When the user sets the store clerk judgment method to judge through the human figure database, each human figure entering / leaving the store will be compared with the data in the human figure database for ReID feature value similarity. Those exceeding the threshold will be judged as store clerks, and no subsequent comparison or merging will be performed; for those judged as non-store clerks, a subsequent comparison and merging process will be carried out.

[0040] Alignment includes "in-out" matching and "in-in" merging:

[0041] "Entry-leave" matching: the total number of store entry events on a certain store on that day is N, and the number of store exit events is M; for store entry event in_i (i = 1 ~ N), the event start time is s_i, and the event end event is o_i; for store exit event out_j (j = 1 ~ M), the event start time is s_j, and the event end event is o_j. At 24:00 on the same day, all store entry events are arranged in positive order from top to bottom according to the "event start time"; all store exit events are also arranged in positive order from left to right according to the "event start time" to form an "entry-leave" matrix table, such as Figure 3 As shown. Eliminate the event combinations whose start time of the leaving event is not after the end time of the entering event, and calculate the ReID similarity of all other entering-leaving events. For the data group whose difference between the start time of the leaving event and the start time of the entering event is less than the matching time, multiply the ReID similarity by a coefficient of 1; for the data group whose difference between the start time of the leaving event and the start time of the entering event is greater than the matching time, multiply the ReID similarity by a coefficient of Z (Z≤1); for data exceeding the similarity threshold X, match "entering events and leaving events" in order from high to low according to the ReID similarity (entering and leaving data that have been matched into data groups will no longer participate in subsequent matching). According to the matching results, a data table with the same partial session_id (entering-leaving matching) is obtained, as shown below. Figure 4 shown.

[0042] like Figure 5 As shown, for example:

[0043] 1. Search the entire table to find the values ​​with similarity ≥ X, and merge the store entry / exit events in descending order of similarity (93>86>80=80>79>75>73=73>88*Z>X);

[0044] 2. The highest similarity is 93. "Enter 2" and "Leave 6" are successfully matched. Set the session_id of Leave 6 = the session_id of Enter 2, and both will no longer participate in subsequent matching;

[0045] 3. The second highest similarity is 86. "Enter N" and "Leave M" are successfully matched. Set the session_id of Leave M = the session_id of Enter N, and both will no longer participate in subsequent matching;

[0046] 4. The next similarity is 80, which are "Enter 1 and Leave 5", "Enter 3 and Leave 5" respectively. When the similarity between one leave event and multiple enter events is equal, this leave event will be matched with the nearest enter event, that is, "Leave 5" and "Enter 3" are matched. Set the session_id of Leave 5 = the session_id of Enter 3, and both will no longer participate in subsequent matching;

[0047] 5. The next similarity is 79. "Enter 3" and "Leave 3" are matched, but "Enter 3" and "Leave 5" have already been matched. Therefore, "Leave 3" is vacant and not matched to an enter event;

[0048] 6. The next similarity is 75. "Enter 2" and "Leave 4" are matched, but "Enter 2" and "Leave 6" have already been matched. Therefore, "Leave 4" is vacant and not matched to an enter event;

[0049] 7. The next similarity is 73, which are "Enter 4 and Leave 7", "Enter 4 and Leave 8" respectively. When the similarity threshold between one enter event and multiple leave events is equal, this enter event will be matched with the nearest leave event, that is, Enter 4 and Leave 7. Set the session_id of Leave 7 = the session_id of Enter 4, and both will no longer participate in subsequent matching;

[0050] 8. The next similarity is 88*Z. "Enter 5" and "Leave M - 1" are successfully matched. Set the session_id of Leave M - 1 = the session_id of Enter 5, and both will no longer participate in subsequent matching;

[0051] 9. Other events with similarity < X are the enter or leave events that are not successfully matched, and the session_id remains unchanged.

[0052] 10. Obtain a data table with some session_ids being the same (enter - leave matching) according to the matching results.

[0053] "In-In" merging: After completing the in-out matching, define the in-store event in_i (i = 1 to N); for the in-store event in_i (i = 1 to N) that has not been matched with an out-store event, its event start time is s_i, and the event end time is o_i; for the in-store event in_i (i = 1 to N) that has been matched with an out-store event (out_j), its overall event start time is s_i, and the overall event end time is o_j. Sort by the event start time, calculate the ReID similarity between in-store events with non-overlapping event times, and obtain the in-store event similarity table, as shown in Figure 5 shown. Merge the "In-In" events in descending order of ReID similarity for data exceeding the similarity threshold Y. When merging meets the merging conditions, assign the session_id of the in-store event with an earlier event start time to the in-store event with a later event start time, and add the event segments of the two for subsequent determination of whether the time periods overlap during merging. The data that has already been merged can participate in subsequent merges. Obtain a data table with the same partial session_id (in-out matching and in-in merging), as shown in Figure 6 shown. Obtain the in-store personnel list and the number of in-store times for each person, as well as the in / out time (and stay duration) for each time.

[0054] As shown in Figure 6 shown, example:

[0055] 1. Full-table retrieval, find values with similarity ≥ Y, and merge the in-store events in descending order of similarity (93 > 86 > 84 > 80 = 80 = Y);

[0056] 2. The highest similarity is 93, "in6" and "in1" are successfully matched, set the session_id of in6 = the session_id of in1, and add the time range of in6 to the time range of in1;

[0057] 3. The second-highest similarity is 88, "inN" and "in2" are successfully matched, set the session_id of inN = the session_id of in2, and add the time range of inN to the time range of in2;

[0058] 4. The next similarity is 86, "inN" and "in6" are matched, set the session_id of inN and the session_id of in2 = the session_id of in1 (because the session_id of in6 = the session_id of in1), and add the time ranges of inN, in6, in2, and in1;

[0059] 5. The next similarity is 80, which are "Entry 5 and Entry 1", "Entry 5 and Entry 4". However, since there is a time conflict between Entry 5 and Entry 6, Entry 5 cannot be merged with Entry 6 and Entry 1. Entry 5 is a separate in-store data entry, and its session_id remains unchanged.

[0060] 6. Other events with a similarity <Y are in-store events that do not meet the merging conditions, and the session_id remains unchanged.

[0061] Based on the merging results, a data table with some identical session_ids (entry-exit matching and entry-entry merging) is obtained. A list of in-store personnel is obtained, along with the number of in-store visits for each person and the entry / exit time (and stay duration) for each visit.

[0062] When the user sets the store clerk judgment method to be based on the number of in-store visits / stay duration, the store clerk data is filtered according to the store clerk judgment rules set by the user.

[0063] Filtering method: By setting the threshold for the number of in-store visits and the threshold for the stay duration, when both the threshold for the number of in-store visits and the threshold for the stay duration are met, it is determined whether it is a store clerk. If it is a store clerk, it is not counted in the passenger flow. If not, the number of customer in-store visits is incremented by 1. Here, the customer can customize the store clerk filtering rules.

[0064] According to the customer deduplication duration set by the user, the data of in-store customers is processed. Multiple in-store visits within the deduplication duration are only counted as one valid in-store customer data entry (i.e., the number of in-store customers + 1).

[0065] Finally, based on the in-store passenger flow data uploaded by the device to the platform and the in-store passenger flow data after the above processing by the platform, they are summarized and displayed within the platform. The platform calculates the in-store passenger flow, in-store records, stay time, and in-store customer portraits.

[0066] The embodiments of this specific implementation manner are all preferred embodiments of the present invention, and do not limit the protection scope of the present invention accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for identifying, comparing, and merging passenger flow statistics using ReID technology, characterized in that: The steps include: A: Roughly count the data of passing through / entering the store through the device, including the algorithm detecting the human figure and tracking the trajectory, and uploading the human figure trajectory status after the trajectory is screened to the cloud; B: The customer data entering the store is filtered in a refined manner through the cloud, and the feature values ​​of the human figure are extracted through ReID. The data is filtered, screened, and deduplicated, and finally matched and merged to obtain the results.

2. According to claim 1, a method for identifying, comparing, and merging passenger flow statistics using ReID technology is characterized in that: Humanoid status judgment: Establish judgment rules for passing through the store, entering the store, and leaving the store on the device to judge the event status of each humanoid.

3. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 1, characterized in that: The human figure image uploaded to the platform by the device is sent to the algorithm model analysis server, and the personnel are screened using the image recognition algorithm model.

4. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 3, characterized in that: The ReID technology is used to extract the feature values ​​of the human figure uploaded to the platform. If multiple human figure images are uploaded, the feature values ​​from the multiple human figure images will be weighted and fused into a set of feature values.

5. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 1, characterized in that: The user sets the rules for the clerk's judgment method: Compare each store entry / exit human figure with the data in the human figure database and compare the ReID feature value similarity: Those who exceed the threshold are judged as store employees and no subsequent comparison or merging is performed; Subsequent comparison and merging process will be carried out for those who are determined to be non-store employees.

6. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 5, characterized in that: The alignment includes "in-out" matching and "in-in" merging.

7. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 1, characterized in that: When the user sets the store clerk judgment method to judge based on the number of visits / length of stay in the store, the store clerk data is filtered according to the store clerk judgment rules set by the user.

8. The method for identifying, comparing, and merging passenger flow statistics using ReID technology according to claim 1, characterized in that: The data of customers entering the store are processed according to the customer deduplication time set by the user. If a customer enters the store multiple times within the deduplication time, only one valid customer data is recorded.

Images