Unmanned store customer tracking method and system

Abstract

The invention relates to an unmanned store customer tracking method and system, and the method specifically comprises the steps: shooting an unmanned store video through a camera of an unmanned store, carrying out the target detection of frames in the video, obtaining a detection frame, obtaining a track of a confirmation state which is matched unsuccessfully last time, carrying out the statistics of the number of times of matching failure of the track of the confirmation state which is matched unsuccessfully, and carrying out the tracking of the customer. Adjusting the weight of the mahalanobis distance and the appearance cosine distance of the track of the matching failure in the current cascade matching based on the number of times of the matching failure; if the track which fails to match fails in the current cascade matching stage and is successfully matched in the IOU matching stage, confirming the track which is successfully matched with the IOU according to the appearance cosine distance, and taking the confirmed track which is successfully matched with the IOU as the track which is successfully matched in the IOU matching stage; and tracking the customer according to the successfully matched tracks in the cascade matching stage and the IOU matching stage. According to the invention, the accuracy of customer tracking can be improved, and the probability of mismatching is reduced.

Description

Technical Field

[0001] The present invention relates to the field of unmanned stores, and in particular to a method and system for tracking customers in unmanned stores. Background Art

[0002] Traditional retail stores rely on manual labor for product management and customer service, while unmanned stores can achieve functions such as self-service shopping, automatic settlement, and loss prevention monitoring through various automation technologies. However, since unmanned stores lack traditional human monitoring, how to effectively track customers to improve their shopping experience and ensure the safety of goods has become a technical problem that needs to be solved urgently. In this context, multi-object tracking (MOT) technology has become an important part of unmanned stores. Multi-object tracking technology also plays an important role in customer behavior analysis. Unmanned stores can collect and analyze customers' shopping behaviors by tracking their activity trajectories in the store, such as the time they stay in each product area and their movement paths. By analyzing this data, stores can better understand customers' shopping habits and preferences, optimize product display layout, and improve shopping experience. For example, stores can adjust the order of product placement according to the length of time customers stay, or place hot-selling products on the paths that customers often pass by to improve sales conversion rates. Through in-depth analysis of customer behavior, unmanned stores can not only improve operational efficiency, but also personalize the configuration of stores to meet the needs of more customers and improve overall service quality. Moreover, this technology can help detect abnormal behaviors in time, such as customers taking goods without paying, thereby improving the store's security protection capabilities. However, in the scenario of unmanned stores, customer behaviors are complex and the environment changes are diverse. When dealing with occlusion, repeated entry and exit of the camera, and other issues, traditional tracking methods may cause tracking drift and ID switching. Improving the tracking accuracy is the key to fully play the role of multi-target tracking. Summary of the invention

[0003] Based on the above problems to be solved, the present invention provides a method for tracking customers in an unmanned store, the method comprising the following steps: The camera of the unmanned store is used to shoot a video of the unmanned store, and the frames in the video are subjected to target detection to obtain the detection frame. The confirmed state trajectory of the last failed match is obtained, and the number of failed matching confirmation state trajectories is counted. Based on the number of failed matching, the weights of the Mahalanobis distance and appearance cosine distance of the failed matching trajectory in the current cascade matching are adjusted; If the track that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, the track that successfully matched IOU is confirmed according to the appearance cosine distance, and the confirmed track that successfully matched IOU is used as the track that successfully matched in the IOU matching stage; Customer tracking is achieved based on the successful matching trajectories in the cascade matching stage and the IOU matching stage.

[0004] Preferably, the weights of the Mahalanobis distance and the appearance cosine distance of the track of failed matching in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

[0005] Preferably, the confirmation of the trajectory of successful IOU matching according to the appearance cosine distance is specifically as follows: Obtain the current position of the trajectory that is successfully matched in the IOU matching stage, and obtain the change amount of the appearance feature in the trajectory that is successfully matched in the cascade within a preset range of the current position; If the appearance cosine distance is not greater than the variation, the IOU matching is deemed to be successful; otherwise, the IOU matching is deemed to have failed.

[0006] Preferably, the acquired trajectory is a cascade of the variation of the appearance features in the trajectory that is successfully matched within a preset range of the current position, specifically: Obtaining the appearance feature vector of the trajectory extracted by ReID and the position corresponding to the appearance feature vector, and if the position is within a preset range of the current position and the cascade matching is successful, adding the appearance feature vector to the appearance feature vector set; The cosine similarity between each appearance feature vector in the appearance feature vector set and a previous appearance feature vector of the same trajectory is calculated, and an average value of all cosine similarities in the appearance feature vector set is calculated, and 1 minus the average value is taken as the variation.

[0007] Preferably, after the customer is tracked according to the track of successful matching in the cascade matching stage and the IOU matching stage, the customer behavior is analyzed; the analysis of the customer behavior is specifically: Record the time when customers enter and leave the store, as well as the time they stay in each merchandise area and the merchandise they purchase; A heat map is used to show the areas with the highest dwell time in the unmanned store and the popular items purchased.

[0008] In addition, the present invention also provides an unmanned store customer tracking system, the system comprising the following modules: The cascade matching module is used to shoot videos of unmanned stores through the cameras of unmanned stores, perform target detection on frames in the videos to obtain detection frames, obtain the confirmed state trajectory of the last failed match, count the number of failed matches of the confirmed state trajectory, and adjust the weights of the Mahalanobis distance and appearance cosine distance of the failed match trajectory in the current cascade matching based on the number of failed matches; The IOU matching module is used to confirm the IOU matching trajectory according to the appearance cosine distance if the trajectory that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, and use the confirmed IOU matching trajectory as the successful matching trajectory in the IOU matching stage; The tracking module is used to track customers according to the successful matching trajectories in the cascade matching stage and the IOU matching stage.

[0009] Preferably, the weights of the Mahalanobis distance and the appearance cosine distance of the track of failed matching in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

[0010] Preferably, the confirmation of the trajectory of successful IOU matching according to the appearance cosine distance is specifically as follows: Obtain the current position of the trajectory that is successfully matched in the IOU matching stage, and obtain the change amount of the appearance feature in the trajectory that is successfully matched in the cascade within a preset range of the current position; If the appearance cosine distance is not greater than the variation, the IOU matching is deemed to be successful; otherwise, the IOU matching is deemed to have failed.

[0011] Preferably, the acquired trajectory is a cascade of the variation of the appearance features in the trajectory that is successfully matched within a preset range of the current position, specifically: Obtaining the appearance feature vector of the trajectory extracted by ReID and the position corresponding to the appearance feature vector, and if the position is within a preset range of the current position and the cascade matching is successful, adding the appearance feature vector to the appearance feature vector set; The cosine similarity between each appearance feature vector in the appearance feature vector set and a previous appearance feature vector of the same trajectory is calculated, and an average value of all cosine similarities in the appearance feature vector set is calculated, and 1 minus the average value is taken as the variation.

[0012] Preferably, after the customer is tracked according to the track of successful matching in the cascade matching stage and the IOU matching stage, the customer behavior is analyzed; the analysis of the customer behavior is specifically: Record the time when customers enter and leave the store, as well as the time they stay in each merchandise area and the merchandise they purchase; A heat map is used to show the areas with the highest dwell time in the unmanned store and the popular items purchased.

[0013] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0014] The present invention dynamically adjusts the weights of the Mahalanobis distance and the appearance cosine distance according to the number of matching failures, thereby reducing the problem of inaccurate position prediction accumulation due to continuous matching failures and reducing incorrect matches caused by trajectory prediction failures. In addition, relying solely on IoU matching can easily lead to mismatching when targets overlap or are close to each other. By confirming the appearance features, it avoids the incorrect association of targets with large visual differences, thereby improving the accuracy of tracking customers in unmanned stores. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of Embodiment 1; Figure 2 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0016] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1, the present invention provides a method for tracking customers in an unmanned store, such as Figure 1 As shown, the method comprises the following steps: S1, use the camera of the unmanned store to shoot the video of the unmanned store, perform target detection on the frames in the video to obtain the detection frame, and obtain the confirmed state trajectory of the last failed match, count the number of failed matching of the confirmed state trajectory, and adjust the weight of the Mahalanobis distance and appearance cosine distance of the failed matching trajectory in the current cascade matching based on the number of failed matching; An unmanned store refers to a store without salespeople or service staff. After entering an unmanned store, users can purchase goods on their own. By tracking customers in unmanned stores, it is possible to understand their shopping habits, and then optimize the displayed goods and the display locations of the goods. The present invention uses the camera of an unmanned store to shoot videos of unmanned stores, and is implemented by tracking customers in the video. When the present invention is implemented, notices will be posted at the door of the unmanned store or users will be informed through an APP, etc., to obtain the consent of the customer. Moreover, if this application involves conflicts with laws and regulations, the present invention shall be implemented within the scope permitted by laws and regulations. In one embodiment, the present invention does not identify personal information of specific customers, such as facial information, etc., but simply numbers customers in the order of entering the store.

[0019] When tracking customers, the DeepSort model is preferably used. First, the target in the video frame is detected, and the target is a customer. If there is a customer in the store, at least one detection frame (Detections) can be obtained by detecting the target in the video frame. The following is to match the detection frame with the existing track (Tracks). The matching includes cascade matching and IOU matching. If the cascade matching fails, the IOU matching will be further performed. However, for the confirmation state track that fails to match, that is, in the cascade matching stage and the IOU matching stage, the confirmation state track and the detection frame are not matched, and the track will not be updated, but the Kalman filter will continue to predict the confirmation state track, which may lead to error accumulation. For example, the first match is not successful, and the predicted position is D1. The second match is not successful, and the predicted position is D2. However, due to the failure of the cascade matching, the predicted position and the position of the detection frame are getting larger and larger, and it is becoming more and more difficult to match, but the appearance of the same customer generally does not change much. Among them, the trajectory of the confirmation state that fails to match is a trajectory that is in the confirmation state and fails in both cascade matching and IOU matching. Cascade matching combines motion information and appearance information to match the trajectory and the detection frame. As the number of failed matching of the confirmed trajectory increases, the motion information can no longer accurately reflect the relationship between the trajectory and the detection frame. The present invention adjusts the weights of the Mahalanobis distance and the appearance cosine distance of the failed matching trajectory in the current cascade matching based on the number of failed matching. Specifically, as the number of failed matching increases, the weight of the appearance cosine distance in the cascade matching is increased, and the weight of the Mahalanobis distance is reduced.

[0020] In a specific embodiment, the weights of the Mahalanobis distance and the appearance cosine distance of the track of failed matching in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

[0021] In DeepSort, there are default appearance cosine distance weights and default Mahalanobis distance weights, and there is a maximum age of the confirmed track. If a confirmed track fails to match successfully for more than the maximum age, the track will be deleted and will not participate in the matching of the track and the detection box in the future. Calculate the ratio of the number of consecutive matching failures to the maximum age. For example, if the number of consecutive matching failures is 3 and the maximum age is 30, the ratio is 0.1. The value obtained by subtracting the ratio from 1 is 0.9. Multiply the value by the default weight of the Mahalanobis distance. If the default Mahalanobis distance weight is 0.5, the weight of the Mahalanobis distance in the current cascade matching is 0.45, and the weight of the appearance cosine distance is 0.55. As the number of matching failures increases, the appearance features play an increasingly important role in cascade matching, that is, the cascade matching is completed more relying on the appearance features.

[0022] The confirmed track that failed to match last time will use the new weights of Mahalanobis distance and appearance cosine distance to calculate the cost in the current cascade matching stage, and the other tracks will still use the default Mahalanobis distance and appearance cosine distance weights to calculate the cost. After all tracks and all detection boxes have completed the cost calculation, the cost matrix is ​​obtained, and then the tracks are matched using the Hungarian algorithm.

[0023] S2, if the trajectory that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, the trajectory that successfully matched IOU was confirmed according to the appearance cosine distance, and the confirmed trajectory that successfully matched IOU was used as the trajectory that successfully matched in the IOU matching stage; If the cascade matching fails, the trajectory will be further matched with IOU. The IOU matching stage only relies on IOU to determine whether it is a match. However, if customers block each other's part or the customer flow is large, mismatching may occur. For example, trajectory A should match detection frame 1, but because detection frame 1 overlaps detection frame 2, it is possible that trajectory A matches detection frame 2. If the confirmed trajectory fails in the cascade matching, it indicates that the appearance features of the confirmed trajectory and the detection frame are quite different. In the IOU matching stage, the match is only determined to be successful based on IOU. Due to the large difference in appearance, its credibility is low. Based on this, the present invention further confirms the successful IOU matching trajectory based on the appearance cosine distance, and uses the confirmed successful IOU matching trajectory as the successful matching trajectory in the IOU matching stage.

[0024] In one embodiment, if the track that failed to match fails to match in the current cascade matching stage but successfully matches in the IOU matching stage, the current position of the track in the IOU matching is obtained. For example, if the position of the track in the IOU matching is (x, y, h, w), then (x, y, h, w) is used as the current position. The preset range is an area centered on the center of the current position. In one embodiment, the length of the preset range is 2h or 3h and the width is 2w or 3w. In another embodiment, the preset range is divided according to the layout of the unmanned store. The preset range is not specifically limited here.

[0025] Then, all trajectories with successful cascade matching within the preset range of the current position are obtained. The so-called trajectory with successful cascade matching within the preset range means that the predicted trajectory is within the preset range and is successfully matched with the detection frame by cascade matching. The appearance feature vector at this time will be stored in the trajectory. Specifically, the appearance feature vector is extracted by ReID in DeepSort. In the present invention, the position of the appearance feature vector in the frame when the appearance feature vector is extracted from the trajectory with successful cascade matching is also recorded, that is, if a trajectory is successfully cascade matched, the appearance feature vector will be extracted by ReID, and the position information of the appearance feature vector in the frame will be recorded. If the position is within the preset range, the appearance feature vector is added to the appearance feature vector set. In one embodiment, the appearance feature vector set is a finite and first-in-first-out set. The change amount of each appearance feature vector in the appearance feature vector set relative to the previous appearance feature vector of the same trajectory is calculated. For example, the 10th appearance feature vector of trajectory 1 is obtained after the cascade matching is successful, and the position in the frame corresponding to the 10th appearance feature vector is within the preset range, then the 10th appearance feature vector of trajectory 1 is added to the appearance feature vector set. In one embodiment, cosine similarity is used as a way to measure the amount of change, and the cosine similarity between each appearance feature vector in the appearance feature vector set and the previous appearance feature vector of the same trajectory is calculated. Still taking the 10th appearance feature vector of trajectory 1 as an example, the previous appearance feature vector of the same trajectory is the 9th appearance feature vector of trajectory 1, and the cosine similarity between the 10th appearance feature vector and the 9th appearance feature vector of trajectory 1 is calculated. Each appearance feature vector corresponds to an area in the frame. In one embodiment, the position of the appearance feature vector is within a preset range of the current position. Preferably, the area in the frame corresponding to the appearance feature vector is completely within the preset range.

[0026] After calculating the cosine similarity corresponding to all appearance feature vectors in the appearance feature vector set, further calculate the average value, and subtract the average value from 1 as the change. If the change is large, it means that within the preset range, the customer turns around, turns his head, squats, or is partially blocked, etc., resulting in a large change in the outside. At this time, the appearance feature vector of the trajectory that is successfully matched in the IOU matching stage is allowed to have a large change, that is, the appearance cosine distance is large. Otherwise, the IOU matching stage is considered unreliable, that is, the trajectory that is successfully matched by the IOU matching stage is considered to be a mismatch. Specifically, if the appearance cosine distance is not greater than the change, the IOU match is considered to be successful, otherwise, the IOU match is considered to have failed. For example, the average value of the cosine similarity calculated by the appearance feature vector set is 0.7, and the change is 0.3; if the appearance cosine distance is 0.2, the IOU match is considered to be successful, and if the appearance cosine distance is 0.4, the IOU match is considered to have failed.

[0027] S3, tracking the customer based on the successful matching trajectories in the cascade matching stage and the IOU matching stage.

[0028] For the unconfirmed trajectories and the confirmed trajectories that were successfully matched last time, the cascade matching method and IOU matching method in DeepSort are continued to be used for operation, and the entire cascade matching and IOU matching are completed together with the confirmed trajectories that failed to match last time to achieve customer tracking.

[0029] In yet another embodiment, S3 further includes analysis of customer behavior: Record the time when customers enter and leave the store, as well as the time they stay in each merchandise area and the merchandise they purchase; A heat map is used to show the areas with the highest dwell time in the unmanned store and the popular items purchased.

[0030] Embodiment 2: The present invention also provides a customer tracking system for unmanned stores, such as Figure 2 As shown, the system includes the following modules: The cascade matching module is used to shoot videos of unmanned stores through the cameras of unmanned stores, perform target detection on frames in the videos to obtain detection frames, obtain the confirmed state trajectory of the last failed match, count the number of failed matches of the confirmed state trajectory, and adjust the weights of the Mahalanobis distance and appearance cosine distance of the failed match trajectory in the current cascade matching based on the number of failed matches; The IOU matching module is used to confirm the IOU matching trajectory according to the appearance cosine distance if the trajectory that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, and use the confirmed IOU matching trajectory as the successful matching trajectory in the IOU matching stage; The tracking module is used to track customers according to the successful matching trajectories in the cascade matching stage and the IOU matching stage.

[0031] Preferably, the weights of the Mahalanobis distance and the appearance cosine distance of the track of failed matching in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

[0032] Preferably, the confirmation of the trajectory of successful IOU matching according to the appearance cosine distance is specifically as follows: Obtain the current position of the trajectory that is successfully matched in the IOU matching stage, and obtain the change amount of the appearance feature in the trajectory that is successfully matched in the cascade within a preset range of the current position; If the appearance cosine distance is not greater than the variation, the IOU matching is deemed to be successful; otherwise, the IOU matching is deemed to have failed.

[0033] Preferably, the acquired trajectory is a cascade of the variation of the appearance features in the trajectory that is successfully matched within a preset range of the current position, specifically: Obtaining the appearance feature vector of the trajectory extracted by ReID and the position corresponding to the appearance feature vector, and if the position is within a preset range of the current position and the cascade matching is successful, adding the appearance feature vector to the appearance feature vector set; The cosine similarity between each appearance feature vector in the appearance feature vector set and a previous appearance feature vector of the same trajectory is calculated, and an average value of all cosine similarities in the appearance feature vector set is calculated, and 1 minus the average value is taken as the variation.

[0034] Preferably, after the customer is tracked according to the track of successful matching in the cascade matching stage and the IOU matching stage, the customer behavior is analyzed; the analysis of the customer behavior is specifically: Record the time when customers enter and leave the store, as well as the time they stay in each merchandise area and the merchandise they purchase; A heat map is used to show the areas with the highest dwell time in the unmanned store and the popular items purchased.

[0035] Embodiment 3, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0036] Embodiment 4: The present invention further provides a computer device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in Embodiment 1 is implemented.

[0037] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A customer tracking method for an unmanned store, characterized in that: The method comprises the following steps: The camera of the unmanned store is used to shoot a video of the unmanned store, and the frames in the video are subjected to target detection to obtain the detection frame. The confirmed state trajectory of the last failed match is obtained, and the number of failed matching confirmation state trajectories is counted. Based on the number of failed matching, the weights of the Mahalanobis distance and appearance cosine distance of the failed matching trajectory in the current cascade matching are adjusted; If the track that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, the track that successfully matched IOU is confirmed according to the appearance cosine distance, and the confirmed track that successfully matched IOU is used as the track that successfully matched in the IOU matching stage; Customer tracking is achieved based on the successful matching trajectories in the cascade matching stage and the IOU matching stage.

2. The method according to claim 1, characterized in that The weights of the Mahalanobis distance and the appearance cosine distance of the failed matching track in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

3. The method according to claim 1, characterized in that The successful IOU matching trajectory is confirmed according to the appearance cosine distance, specifically: Obtain the current position of the trajectory that is successfully matched in the IOU matching stage, and obtain the change amount of the appearance feature in the trajectory that is successfully matched in the cascade within a preset range of the current position; If the appearance cosine distance is not greater than the variation, the IOU matching is deemed to be successful; otherwise, the IOU matching is deemed to have failed.

4. The method according to claim 3, characterized in that The acquired trajectory is a variation of the appearance features of the trajectory that is successfully cascade matched within the preset range of the current position, specifically: Obtaining the appearance feature vector of the trajectory extracted by ReID and the position corresponding to the appearance feature vector, and if the position is within a preset range of the current position and the cascade matching is successful, adding the appearance feature vector to the appearance feature vector set; The cosine similarity between each appearance feature vector in the appearance feature vector set and a previous appearance feature vector of the same trajectory is calculated, and an average value of all cosine similarities in the appearance feature vector set is calculated, and 1 minus the average value is taken as the variation.

5. The method according to claim 1, characterized in that After the customer is tracked according to the track of successful matching in the cascade matching stage and the IOU matching stage, the customer behavior is analyzed; the analysis of the customer behavior is specifically as follows: Record the time when customers enter and leave the store, as well as the time they stay in each merchandise area and the merchandise they purchase; A heat map is used to show the areas with the highest dwell time in the unmanned store and the popular items purchased.

6. An unmanned store customer tracking system, characterized in that: The system includes the following modules: The cascade matching module is used to shoot videos of unmanned stores through the cameras of unmanned stores, perform target detection on frames in the videos to obtain detection frames, obtain the confirmed state trajectory of the last failed match, count the number of failed matches of the confirmed state trajectory, and adjust the weights of the Mahalanobis distance and appearance cosine distance of the failed match trajectory in the current cascade matching based on the number of failed matches; The IOU matching module is used to confirm the IOU matching trajectory according to the appearance cosine distance if the trajectory that failed to match failed to match in the current cascade matching stage but successfully matched in the IOU matching stage, and use the confirmed IOU matching trajectory as the successful matching trajectory in the IOU matching stage; The tracking module is used to track customers according to the successful matching trajectories in the cascade matching stage and the IOU matching stage.

7. The system according to claim 6, characterized in that The weights of the Mahalanobis distance and the appearance cosine distance of the failed matching track in the current cascade matching are adjusted based on the number of failed matching, specifically: Get the default weight of the Mahalanobis distance in cascade matching, and the maximum lifespan of the confirmed trajectory. The maximum lifespan refers to the maximum number of times that the confirmed trajectory is allowed to fail to match successfully within a number of consecutive frames. After exceeding the maximum number, the confirmed trajectory will be considered invalid and deleted; Calculate the ratio of the number of matching failures to the maximum lifespan number, calculate a value obtained by subtracting the ratio from 1, use the product of the value and the default weight of the Mahalanobis distance as the weight of the Mahalanobis distance in the current cascade matching, and use the difference between 1 and the weight of the Mahalanobis distance in the current cascade matching as the weight of the appearance cosine distance in the current cascade matching.

8. The system according to claim 6, characterized in that The successful IOU matching trajectory is confirmed according to the appearance cosine distance, specifically: Obtain the current position of the trajectory that is successfully matched in the IOU matching stage, and obtain the change amount of the appearance feature in the trajectory that is successfully matched in the cascade within a preset range of the current position; If the appearance cosine distance is not greater than the variation, the IOU matching is deemed to be successful; otherwise, the IOU matching is deemed to have failed.

9. The system according to claim 8, characterized in that The acquired trajectory is a variation of the appearance features of the trajectory that is successfully cascade matched within the preset range of the current position, specifically: Obtaining the appearance feature vector of the trajectory extracted by ReID and the position corresponding to the appearance feature vector, and if the position is within a preset range of the current position and the cascade matching is successful, adding the appearance feature vector to the appearance feature vector set; The cosine similarity between each appearance feature vector in the appearance feature vector set and a previous appearance feature vector of the same trajectory is calculated, and an average value of all cosine similarities in the appearance feature vector set is calculated, and 1 minus the average value is taken as the variation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.

Images