Passenger flow statistical analysis system and method based on rid assistance

Abstract

The invention relates to the technical field of passenger flow statistical analysis, and particularly discloses a passenger flow statistical analysis system and method based on rid assistance. Each piece of human body target data is analyzed by using an image analysis and recognition algorithm based on artificial intelligence and machine vision, so that saliency feature representation about a human body target in each piece of human body target data is captured, and human body orientation judgment is facilitated to obtain a human body orientation recognition result; comprising front, back and side surfaces. Therefore, behavior pattern recognition can be carried out on each human body target object at the door of the shop, so that orientation recognition and detection are carried out on the human body target object, subsequent passenger flow statistical analysis is facilitated, and powerful support is provided for commercial decision making.

Description

Technical Field

[0001] The present application relates to the technical field of passenger flow statistics analysis, and more specifically, to a passenger flow statistics analysis system and method based on ReID assistance. Background Art

[0002] With the continuous development of the retail and service industries, merchants are paying more and more attention to improving customer experience and store operation efficiency. In order to better understand customer behavior patterns, optimize store layout and staffing, accurate customer flow statistics and analysis have become particularly important. Traditional customer flow counting methods usually rely on simple sensors (such as infrared or access control counters), which can only provide rough statistics on the number of people entering and leaving, and cannot distinguish different types of customer behaviors, such as wandering, entering or leaving. In addition, the lack of experience in counting applications for personnel re-identification has led to insufficient granularity in customer flow analysis, making it difficult to obtain accurate results, especially ignoring the method of accurately judging real customers, making it difficult to effectively eliminate non-customer personnel (such as takeaways, staff, etc.).

[0003] Therefore, an optimized passenger flow statistics analysis solution is desired. Summary of the invention

[0004] The present application provides a passenger flow statistics analysis system and method based on ReID assistance, which can recognize the behavior pattern of each human target object at the door of a store, thereby identifying and detecting its direction, which is helpful for subsequent passenger flow statistics analysis and provides strong support for business decision-making.

[0005] In a first aspect, a passenger flow statistical analysis method based on reid assistance is provided, comprising:

[0006] Obtaining door scene monitoring images collected by cameras deployed at the store entrance;

[0007] Performing human target detection on the door scene monitoring image to obtain a set of human target data;

[0008] Using the trained ReID auxiliary model, human body orientation recognition is performed on each human body target data in the set of human body target data to obtain a set of human body orientation recognition results;

[0009] If the human body orientation recognition result is positive, the human body target data is regarded as a person entering the store and the customer flow statistics count is increased by one;

[0010] If the human body orientation recognition result is sideways, the human body target data is regarded as a wandering person;

[0011] If the human body orientation recognition result is the back, the human body target data is regarded as a person leaving, and the passenger flow statistics count is reduced by one.

[0012] In a second aspect, a passenger flow statistics analysis system based on reid assistance is provided, including:

[0013] The door scene monitoring image acquisition module is used to acquire the door scene monitoring image collected by the camera deployed at the door of the store;

[0014] A human target detection module, used for performing human target detection on the door scene monitoring image to obtain a set of human target data;

[0015] A human body orientation recognition module, used for performing human body orientation recognition on each human body target data in the set of human body target data by using the trained ReID auxiliary model to obtain a set of human body orientation recognition results;

[0016] A store entry personnel statistics module, configured to, if the human body orientation recognition result is positive, regard the human body target data as a store entry person and increase the customer flow statistics count by one;

[0017] a wandering person determination module, configured to regard the human target data as a wandering person if the human body orientation recognition result is sideways;

[0018] The leaving person statistics module is used to regard the human target data as a leaving person if the human body direction recognition result is the back, and the passenger flow statistics count is reduced by one.

[0019] The present application provides a passenger flow statistics analysis system and method based on ReID assistance. After human target detection is performed on the collected door scene monitoring images, each human target data is analyzed using an image analysis and recognition algorithm based on artificial intelligence and machine vision, so as to capture the significant feature representation of the human target in each human target data, which is helpful for human orientation discrimination to obtain human orientation recognition results, including front, back and side. In this way, the behavior pattern of each human target object at the door of the store can be recognized, so as to identify and detect its orientation, which is helpful for subsequent passenger flow statistics analysis and provides strong support for business decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0021] Figure 1 This is a schematic flow chart of a passenger flow statistical analysis method based on reid assistance according to an embodiment of the present application.

[0022] Figure 2This is a data flow diagram of the passenger flow statistical analysis method based on reid assistance in an embodiment of the present application.

[0023] Figure 3 This is a schematic flowchart of S3 in the passenger flow statistical analysis method based on reid assistance in an embodiment of the present application.

[0024] Figure 4 This is a schematic flowchart of S31 in the passenger flow statistical analysis method based on reid assistance according to an embodiment of the present application.

[0025] Figure 5 This is a schematic flowchart of S32 in the passenger flow statistical analysis method based on reid assistance in an embodiment of the present application.

[0026] Figure 6 This is a schematic flowchart of S323 in the passenger flow statistical analysis method based on reid assistance in an embodiment of the present application.

[0027] Figure 7 This is a schematic block diagram of a passenger flow statistics analysis system based on re-ID assistance according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0029] In recent years, with the development of computer vision technology and deep learning algorithms, especially the progress of pedestrian re-identification (Re-Identification, ReID) technology, more accurate passenger flow statistics have become possible. Re-ID technology focuses on identifying the same pedestrian from different camera perspectives, even if the pedestrian's appearance changes due to factors such as angle, lighting, and occlusion. This technology can be used to track and identify specific individuals in a monitoring environment, thereby achieving more detailed and accurate passenger flow analysis.

[0030] Based on this, in the technical solution of this application, a passenger flow statistics analysis method based on re-ID assistance is proposed, which can combine target detection and improved personnel re-identification technology to perform accurate crowd counting to support passenger flow statistics. Specifically, Figure 1 and Figure 2As shown, in the above-mentioned passenger flow statistics analysis method based on re-ID assistance, the specific implementation steps are: S1, obtaining the door scene monitoring image collected by the camera deployed at the door of the store; S2, performing human target detection on the door scene monitoring image to obtain a set of human target data; S3, using the trained re-ID auxiliary model to perform human orientation recognition on each human target data in the set of human target data to obtain a set of human orientation recognition results; S4, if the human orientation recognition result is the front, the human target data is regarded as a person entering the store and the passenger flow statistics count is increased by one; S5, if the human orientation recognition result is the side, the human target data is regarded as a wandering person; S6, if the human orientation recognition result is the back, the human target data is regarded as a person leaving, and the passenger flow statistics count is reduced by one. In addition, the passenger flow statistics analysis method based on re-ID assistance also includes the steps: S7, performing object re-recognition on each human target data in the set of human target data to obtain a set of object re-recognition results; S8, in response to the object re-recognition result being the same person entering the store, the human target data is not repeatedly counted.

[0031] Exemplarily, in step S1, a monitoring image of the door scene collected by a camera deployed at the door of a shop is obtained. It should be understood that the purpose of obtaining the monitoring image of the door scene collected by the camera deployed at the door of the shop is to achieve accurate passenger flow statistics analysis. In this way, all human target objects entering and leaving the shop can be captured, and their behavior patterns can be recognized, so as to accurately determine whether they are customers entering the store, wandering or leaving. This helps merchants to better understand the behavioral intentions and flow changes of customers, optimize store layout and service configuration, and improve operational efficiency. By analyzing the monitoring image of the door scene, the specific behavior pattern of the customer can be determined according to the human body orientation (front, side or back), such as whether they actually enter the store or just wander at the door. Traditional people counting methods are easily affected by environmental factors, resulting in inaccurate data. Through image analysis technology, the problem of repeated counting can be effectively avoided, and non-customer personnel (such as takeaways, staff) can be filtered out to provide more realistic passenger flow information. Based on accurate passenger flow statistics results, merchants can obtain valuable information about customer visit time, frequency, etc., so as to make more reasonable business decisions, such as adjusting business hours, arranging promotional activities, etc.

[0032] In one embodiment, in order to obtain the door scene monitoring image, it is first necessary to install a high-definition camera at the store door to ensure that the entire entrance and exit area is covered so as to fully record all human targets entering and leaving. The camera should have features such as night vision and wide-angle lens to meet the monitoring needs under different lighting conditions. Then the camera is connected to the local server or cloud platform to ensure that the image data can be transmitted and stored in real time, and at the same time ensure that the network bandwidth is stable enough to prevent image loss or delay due to network problems.

[0033] Exemplarily, in step S2, human target detection is performed on the door scene monitoring image to obtain a set of human target data. It should be understood that the purpose of performing human target detection on the door scene monitoring image to obtain a set of human target data is to accurately identify all human objects entering and leaving the store from a complex background environment. This process is a key step in passenger flow statistical analysis. It can not only distinguish customers from non-customers (such as deliverymen and staff), but also provide necessary basic information for subsequent behavior pattern recognition and direction discrimination. Through human target detection, the specific characteristics of each individual, such as position, posture, etc., can be captured, which helps to track the individual's behavior trajectory more accurately and make more detailed business decisions based on this, such as optimizing store layout, improving service quality, arranging promotional activities, etc.

[0034] In one embodiment, the present application uses an anchor box-based target detection network to perform human target detection. Specifically, the entire process is implemented by a specially trained personnel target detection network, which can effectively filter out irrelevant features and focus on identifying human targets. Specifically, the door scene monitoring image is subjected to human target detection to obtain a set of human target data, including the following steps: First, the acquired door scene monitoring image is input into the personnel target detection network. This network is an anchor window-based target detection model, such as Fast R-CNN, Faster R-CNN or RetinaNet, etc., which all have powerful target positioning capabilities. Then, the target anchoring layer in these models is used, that is, a series of anchor boxes B of predefined sizes and proportions are used to perform sliding window scanning on the image. Each anchor box B will try to match the human targets that may exist in the image according to its location and generate the corresponding candidate area. When the anchor box B slides on the image, it not only frames the potential human target interest area, but also evaluates the possibility that these areas contain actual human bodies. For each candidate region that is considered to be a human target, the network will output a set of coordinate information to define the position of the human body in the region, as well as a confidence score that indicates the reliability of this prediction. In order to improve the quality of the detection results, the non-maximum suppression (NMS) algorithm is usually implemented to eliminate redundant overlapping detection boxes to ensure that the final result is the most accurate and non-repetitive human target data set. After this processing, all candidate regions that are considered to contain human bodies constitute a set of human target data, providing a reliable basis for subsequent analysis.

[0035] Exemplarily, in step S3, the trained ReID auxiliary model is used to perform human orientation recognition on each human target data in the set of human target data to obtain a set of human orientation recognition results. In particular, the process of performing human orientation recognition on each human target data in the set of human target data using the trained ReID auxiliary model is crucial, because different behavior patterns of human targets can be distinguished through human orientation recognition. For example, a person facing forward may be more likely to be a customer entering the store, while a person facing backward may be a customer leaving the store. A person facing sideways may be wandering at the door and has not yet decided whether to enter the store. This distinction helps merchants understand the behavioral intentions of customers more accurately. Simply relying on the number of people entering and leaving the store cannot provide enough information to determine whether a person has truly become a customer. Orientation recognition can help filter out pedestrians who only pass by the store but do not enter, thereby improving the accuracy of passenger flow statistics.

[0036] Based on this, in the process of human orientation recognition for each human target data in the set of human target data, the technical concept of the present application is to perform human target detection on the collected door scene monitoring images, and then use image analysis and recognition algorithms based on artificial intelligence and machine vision to analyze each human target data, so as to capture the significant feature representation of the human target in each human target data, which is helpful for human orientation discrimination to obtain human orientation recognition results, including front, back and side. In this way, the behavior pattern of each human target object at the door of the store can be recognized, so as to identify and detect its orientation, which is helpful for subsequent passenger flow statistical analysis and provides strong support for business decision-making.

[0037] In one embodiment, Figure 3 As shown, the trained ReID auxiliary model is used to perform body orientation recognition on each human target data in the set of human target data to obtain a set of human orientation recognition results, including: S31, inputting the human target data into a human multi-scale feature extractor based on a dynamic convolutional labeling model to obtain a human target multi-scale image feature coding map; S32, performing feature selection on the human target multi-scale image feature coding map to obtain a human target image significant feature coding map; S33, performing body orientation discrimination based on the human target image significant feature coding map to obtain the human orientation recognition result.

[0038] Exemplarily, in step S31, the human target data is input into a human multi-scale feature extractor based on a dynamic convolutional labeling model to obtain a human target multi-scale image feature encoding map. It should be understood that in actual passenger flow statistics, the camera may capture human targets at different distances and angles. For example, the camera at the door may capture pedestrians near and far at the same time. Therefore, in order to be able to analyze the human target data more comprehensively and carefully, so as to better capture the behavior pattern of the human target data, in the technical solution of the present application, the human multi-scale feature extractor based on the dynamic convolutional labeling model can combine the advantages of CNN and Transformer, and can not only capture the detailed features of the local image blocks of the human target data, but also learn the global semantic information, which enables the system to better adapt to various environmental changes and improve the generalization ability, thereby providing more accurate data support in complex passenger flow statistics scenarios.

[0039] In one embodiment, Figure 4As shown, the human target data is input into a human multi-scale feature extractor based on a dynamic convolutional labeling model to obtain a human target multi-scale image feature coding map, including: S311, performing local image block segmentation on the human target data to obtain a set of human target local image blocks; S312, inputting each human target local image block in the set of human target local image blocks into a human target local semantic feature extractor based on CNN to obtain a set of human target local feature maps; S313, splicing the set of human target local feature maps into a human target global representation feature map and then inputting the human target feature global association encoder based on the Transformer model to obtain the human target multi-scale image feature coding map.

[0040] Specifically, the process of inputting human target data into the human multi-scale feature extractor based on the dynamic convolutional labeling model to obtain the multi-scale image feature encoding map of the human target is a continuous and comprehensive operation. This process starts with the local image block segmentation of the detected human target data, that is, dividing the entire human target area into several small blocks, each of which represents a part of the human body, such as the head, shoulders or arms. This segmentation method helps to capture the detailed information of different parts and provide basic materials for subsequent feature extraction. Subsequently, each of these segmented local image blocks is sent to the human target local semantic feature extractor based on the convolutional neural network (CNN). Through training, the extractor can learn rich semantic features such as texture, color and shape from each local image block. The output result is a series of local feature maps, each of which corresponds to a specific local area, so that the system can understand the uniqueness of each local area more carefully and enhance the adaptability to complex scenes. Next, all local feature maps are spliced ​​together to form a global representation feature map of the human target. This is not just a simple splicing, but a strategy (such as maximum pooling, average pooling or self-attention mechanism) is used to integrate the relationship between local features, so as to construct a global feature representation containing overall structural information. This global representation feature map not only retains local details, but also introduces spatial correlation, making the features richer and more representative. In order to further enhance the feature expression ability, the global correlation encoder of human target features based on the Transformer model is used to process the spliced ​​global representation feature map. At this stage, the dynamic convolution technology is used to adjust the size and shape of the convolution kernel according to different inputs to achieve feature capture of human targets at different scales. At the same time, combined with the multi-scale feature fusion method, the information from different levels is combined to generate the final multi-scale image feature encoding map of the human target. Dynamic convolution and multi-scale feature fusion ensure that the model can work stably under various distance and angle conditions, and improve the robustness and generalization performance of feature extraction.

[0041] Exemplarily, in the step S32, feature selection is performed on the multi-scale image feature coding map of the human target to obtain a significant feature coding map of the human target image. It should be understood that the multi-scale image feature coding map of the human target contains multi-scale coding features related to the human target. However, not all extracted multi-scale features of the human target are helpful for the final human orientation discrimination and passenger flow statistics analysis tasks. Some features may carry redundant information or be irrelevant to the task. Removing these irrelevant features can reduce noise interference, thereby improving the recognition accuracy of the model. Based on this, in order to more effectively identify and highlight the key feature semantics and significant features of the human target, reduce the amount of data for subsequent processing, save computing resources, and provide a basis for subsequent real-time passenger flow statistics, in the technical solution of the present application, feature selection is further performed on the multi-scale image feature coding map of the human target to obtain a significant feature coding map of the human target image. In particular, through the process of feature selection, the feature representation in the deep learning model can be optimized by evaluating the interaction between the local features of the human target and their neighboring local features, so as to achieve effective compression and sparseness of the original multi-scale image feature coding map of the human target.

[0042] In one embodiment, Figure 5 As shown, feature selection is performed on the human target multi-scale image feature coding map to obtain a human target image salient feature coding map, including: S321, feature decomposition is performed on the human target multi-scale image feature coding map to obtain a set of human target multi-scale image semantic coding local feature vectors; S322, each human target multi-scale image semantic coding local feature vector in the set of human target multi-scale image semantic coding local feature vectors is input into an importance measurement module to obtain a set of human target multi-scale image semantic coding local feature importance score values; S323, based on the set of human target multi-scale image semantic coding local feature importance score values, feature selection and feature shape reshaping are performed on the set of human target multi-scale image semantic coding local feature vectors to obtain the human target image salient feature coding map.

[0043] In one embodiment, in step S321, feature decomposition is performed on the multi-scale image feature coding map of the human target to obtain a set of local feature vectors of multi-scale image semantic coding of the human target, including: feature decoupling and feature flattening are performed on the multi-scale image feature coding map of the human target to obtain a set of local feature vectors of multi-scale image semantic coding of the human target. Specifically, the process can be expressed by a formula as follows:

[0044]

[0045]

[0046] in, is the multi-scale image feature coding map of the human target, is a feature decoupling operation, They represent the first, second, and third matrices in the set of multi-scale image semantic coding feature matrices of human targets. and The semantic encoding feature matrix of multi-scale images of human targets, To flatten the features, They represent the first, second, and third local feature vectors in the set of multi-scale image semantic coding of human targets. and Local feature vectors of semantic encoding of multi-scale images of human objects.

[0047] It should be understood that after the multi-scale image feature encoding map of the human target extracted from the surveillance image is initially processed, it contains rich information, but not all information is useful for subsequent recognition tasks. Therefore, through feature decomposition, the complex global feature map can be decoupled into multiple local feature vectors, each of which represents the detailed features of a part of the human target (such as the head, shoulders, arms, etc.). This decomposition enables the system to capture the changes in different parts of the human body in more detail, especially for human targets with different postures, local features often reflect the uniqueness of the individual better than the overall shape.

[0048] In the step S322, each human target multi-scale image semantic coding local feature vector in the set of human target multi-scale image semantic coding local feature vectors is input into an importance measurement module to obtain a set of human target multi-scale image semantic coding local feature importance score values. Specifically, the process can be expressed as follows:

[0049]

[0050] in, and They are The corresponding weight matrix and bias vector, is matrix multiplication, is the modulation vector, for The corresponding human target multi-scale image semantic encoding local feature importance score value.

[0051] Next, the set of local feature vectors of the multi-scale image semantic encoding of the human target obtained above is sent to the importance measurement module. The function of this module is to assign a numerical "importance score" to each local feature vector to measure the weight of the feature in describing the entire individual. Specifically, those features that carry more identity information (such as face) are usually considered more important; while relatively minor features have lower scores. In this way, the system can select the most discernible part from many possible feature combinations and reduce the interference caused by redundant information. After introducing the importance measurement mechanism, it can not only significantly reduce the data dimension and speed up the calculation, but also improve the reliability of the final result. Because in practical applications, not all detected features are helpful for recognition tasks, some may be noise or common features similar to other individuals. By quantifying the importance of each feature, the system can focus more on those unique marks that really help to distinguish different people, thereby improving the accuracy of re-identification. At the same time, this also means that even in the case of low quality or partial occlusion, as long as enough important features are retained, a relatively reliable match can still be achieved.

[0052] In one embodiment, Figure 6 As shown, in the step S323, based on the set of importance score values ​​of the local features of the multi-scale image semantic coding of the human target, the set of local feature vectors of the multi-scale image semantic coding of the human target is subjected to feature selection and feature shape reshaping to obtain the significant feature coding map of the human target image, including: S3231, based on the set of importance score values ​​of the local features of the multi-scale image semantic coding of the human target, the set of local feature vectors of the multi-scale image semantic coding of the human target is arranged in descending order to obtain a descending sequence of local feature vectors of the multi-scale image semantic coding of the human target. Specifically, the process can be expressed by the formula:

[0053]

[0054] in, Indicates descending sort operation. They represent the first, second, and third in the descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target. and Local feature vectors of semantic encoding of multi-scale images of human objects.

[0055] It should be understood that each local feature vector is sorted in descending order according to its importance score value, forming a descending sequence of local feature vectors of multi-scale image semantic encoding of human targets sorted by importance. This operation ensures that the features that best represent the identity of the individual are given priority, while those relatively unimportant features are ranked behind.

[0056] S3232, based on the neighborhood features of each human target multi-scale image semantic coding local feature vector in the descending sequence of the human target multi-scale image semantic coding local feature vector, calculate the feature neighborhood activity of each human target multi-scale image semantic coding local feature vector to obtain a sequence of feature neighborhood activity. Specifically, the process can be expressed by the formula:

[0057]

[0058] in, is the first in the descending sequence of the local feature vector of the multi-scale image semantic encoding of the human target The first local feature vector of semantic coding of multi-scale image of human target Position feature values, For the said The number of eigenvalues ​​in the local feature vector of the semantic encoding of the multi-scale image of the human target, For the said Neighborhood feature factors of local feature vectors of multi-scale image semantic encoding of human targets, It is Neighborhood feature factors of local feature vectors of multi-scale image semantic encoding of human targets, It is the Feature neighborhood activity of local feature vectors of semantic encoding of multi-scale images of human targets.

[0059] That is, for the feature vectors that have been sorted by importance, we further analyze their correlation and distribution, that is, calculate the "feature neighborhood activity" of each local feature vector. Specifically, this refers to evaluating the activity of other related features around a local feature to measure the uniqueness and representativeness of the local feature in the entire image.

[0060] S3233, based on the sequence of the feature neighborhood activity, feature selection is performed on the descending sequence of the semantically encoded local feature vectors of the multi-scale image of the human target to obtain a descending sequence of the semantically encoded local feature vectors of the multi-scale image of the human target after selection. Specifically, the process can be expressed by a formula as follows:

[0061]

[0062]

[0063] in, is a predetermined threshold, is the feature selection process, is the first, second, and third in the descending sequence of the local feature vector of the multi-scale image semantic encoding of the selected human target. and After selection, the local feature vector of multi-scale image semantic encoding of human targets is constructed.

[0064] That is, based on the sequence of feature neighborhood activity, the feature vectors are selected in descending order, with the goal of removing redundant information and retaining those feature vectors that are both important and highly active. This step checks whether each feature vector meets the preset threshold condition, and only those features that meet the criteria will be retained in the final selection list.

[0065] S3234, reshaping the descending sequence of the selected local feature vectors of the multi-scale image semantic coding of the human target to obtain a significant feature coding map of the human target image. Specifically, the process can be expressed as follows:

[0066]

[0067] in, It is the characteristic shape reshaping process, It is the significant feature coding map of the human target image.

[0068] That is, the selected feature vectors are reorganized into new feature encoding graphs, a process called feature reshaping. In this way, the originally dense feature representation is converted into a sparse form, retaining only the most core parts. The sparse feature encoding graph is not only smaller in size, but also easier to store and transmit, and can better adapt to the deployment requirements on different hardware platforms.

[0069] That is, in one embodiment, based on the sequence of feature neighborhood activity, feature selection is performed on the descending sequence of local feature vectors of multi-scale image semantic coding of human target to obtain a descending sequence of local feature vectors of multi-scale image semantic coding of human target after selection, including: extracting the feature neighborhood activity corresponding to the first local feature vector of multi-scale image semantic coding of human target from the sequence of feature neighborhood activity to obtain the first feature neighborhood activity; comparing the first feature neighborhood activity with a predetermined threshold, and in response to the first feature neighborhood activity being greater than or equal to the predetermined threshold, retaining the first local feature vector of multi-scale image semantic coding of human target, otherwise deleting the first local feature vector of multi-scale image semantic coding of human target.

[0070] Specifically, the process of feature selection for the multi-scale image feature coding map of the human target can realize the effective compression and optimization of the input multi-scale image semantic coding feature map of the human target through multi-level analysis of the human target features, which not only improves the efficiency of feature extraction, but also enhances the understanding of the internal structure of the human target, promotes the learning of deeper human target semantics and better generalization ability. Compared with the traditional feature selection algorithm, the feature selection process can identify and remove those features that are important but highly correlated with their neighbors, thereby effectively reducing redundant information and improving model efficiency. At the same time, by comprehensively considering the relationship between the local features of the human target and its neighborhood, it can better capture the complex patterns in the data and the interaction between the local features of the multi-scale image of the human target. In addition, since the feature selection processing method based on importance measurement can emphasize the relationship between features rather than simply relying on the importance of individual features, the selected features often better reflect the true characteristics of the human target semantics, thereby improving the transparency and interpretability of the model decision, and facilitating the understanding of the influence of each local feature of the human target on the final feature representation and subsequent human orientation recognition tasks, thereby improving the interpretability and credibility of the model. In this way, real customers and other non-customers (such as deliverymen and staff) can be distinguished more accurately, providing more in-depth customer behavior analysis and supporting merchants to make better business decisions.

[0071] Exemplarily, in the step S33, the human body orientation is discriminated based on the significant feature coding map of the human body target image to obtain the human body orientation recognition result. In one embodiment, the human body orientation is discriminated based on the significant feature coding map of the human body target image to obtain the human body orientation recognition result, including: inputting the significant feature coding map of the human body target image into a human body orientation discriminator based on a classifier to obtain the human body orientation recognition result. That is to say, the human body target image features after feature selection significance enhancement are used for classification processing, so as to discriminate the human body orientation to obtain the human body orientation recognition result, including the front, back and side. It is worth mentioning that here, the front represents the customer entering the store, the side represents the passerby, and the back represents the customer leaving. In this way, the behavior pattern recognition can be performed on each human target object at the door of the store, so as to identify and detect its orientation, which is helpful for the subsequent passenger flow statistical analysis and provides strong support for business decision-making.

[0072] In one embodiment, the classifier-based human body orientation discriminator uses a support vector machine (SVM). SVM is a supervised learning algorithm that is particularly good at handling binary or multi-classification problems in high-dimensional space. It maximizes the interval between different categories by finding the optimal hyperplane, thereby achieving better generalization performance. In this application, three categories of SVM can be set, corresponding to the three orientations of front, side and back.

[0073] Here, since the multi-scale image feature coding map of the human target represents the multi-scale image semantic coding features of the human posture features of the human target data, when performing feature selection based on the feature neighborhood activity, the feature population attributes of different local features of human posture will have feature selection fairness differences based on the difference in feature neighborhood activity measurement, thereby affecting the micro-macro representation bias of the significant feature coding map of the human target image, and reducing the accuracy of the human orientation recognition results obtained by the classifier-based human orientation discriminator.

[0074] In a preferred example, when the human target image salient feature coding map is input into a human orientation discriminator based on a classifier to obtain a human orientation recognition result, the human target image salient feature coding map is optimized, and the optimization process includes:

[0075] Expanding the human target image salient feature coding map into a human target image salient feature coding vector;

[0076] Based on the feature value of the significant feature coding vector of the human target image Distance and Distance matrix and the distance matrix of the salient feature encoding of human target image ;

[0077] Determine the number of super-distributed eigenvalues ​​in the significant feature coding vector of the human target image whose difference from the eigenvalue mean is greater than the eigenvalue variance, and respectively calculate the reciprocal of the logarithm of the number of super-distributed eigenvalues ​​with base two and the exponential value of the reciprocal of the number of super-distributed eigenvalues ​​with base natural constants to obtain the first human target image significant feature coding full network interactive representation value and the second human target image salient feature encoding full network interactive representation value :

[0078]

[0079]

[0080]

[0081] in, represents the significant feature encoding vector of the human target image, The first eigenvalues, Representation calculation The number of is the significant feature encoding vector of the human target image The number of eigenvalues ​​of the superdistribution in , and and is the significant feature encoding vector of the human target image The mean and variance of all eigenvalues ​​of ;

[0082] Calculate the first human target image salient feature encoding hidden basic feature vector and the second human target image salient feature encoding hidden basic feature vector respectively with the reciprocal of the first human target image salient feature encoding full network interactive representation value and the reciprocal of the second human target image salient feature encoding full network interactive representation value as indexes:

[0083]

[0084]

[0085] in, Represents the first human target image salient feature encoding hidden basic feature vector, Representing a second human target image salient feature encoding hidden basic feature vector;

[0086] The first human target image salient feature encoding hidden basic feature vector as a row vector is matrix-multiplied with the human target image salient feature encoding-distance matrix to obtain a human target image salient feature encoding-distance query vector ,in, Represents matrix multiplication;

[0087] The human target image salient feature encoding two distance matrix is ​​matrix-multiplied with the second human target image salient feature encoding hidden basic feature vector as a column vector to obtain the human target image salient feature encoding two distance response vector ;

[0088] The weighted sum of the distance query vector of the human target image salient feature encoding 1 and the distance response vector of the human target image salient feature encoding 2 is calculated to obtain an optimized human target image salient feature encoding vector. Finally, the optimized human target image salient feature encoding vector is passed through a human orientation discriminator based on a classifier to obtain a human orientation recognition result.

[0089] That is, in order to solve the problem of insufficient overall active query coding performance of the significant feature coding vector of the human target image obtained by expanding the significant feature coding of the human target image due to the long distance exceeding the local spacing limit under the fixed numerical sequence layout, a multi-dimensional attribute basic form based on the self-distance matrix integration of the significant feature coding vector of the human target image is adopted to capture the complex architecture of the whole network interaction of its value domain, and the performance of the live sales temporal semantic query response coding vector is reconstructed by simulating the large-scale multi-dimensional attribute hidden basis of the significant feature coding vector of the human target image, so as to realize the reconstruction of the active search performance connection of the significant feature coding vector of the human target image, and then reflect the predictable reconstruction of the actual series arrangement action of the significant feature coding vector of the human target image, and improve the accuracy of the human orientation recognition result obtained by the significant feature coding vector of the human target image through the classifier-based human orientation discriminator.

[0090] Exemplarily, in step S4, if the human body orientation recognition result is positive, the human body target data is regarded as a person entering the store and the passenger flow statistics count is increased by one. It should be understood that in actual scenarios, a person facing the store entrance usually means that they have a higher probability of entering the store. People tend to face the direction they are going to, so people facing the store door are more likely to be potential customers preparing to enter the store. Using human body orientation recognition technology, real customers and other pedestrians can be distinguished more accurately, thereby improving the accuracy of passenger flow statistics. Accurate passenger flow statistics are crucial for merchants, as they can help optimize store layout, staffing, and service quality. By counting people facing the front, more realistic customer flow information can be obtained, thereby supporting better business decision making.

[0091] Exemplarily, in step S5, if the human body orientation recognition result is sideways, the human body target data is regarded as a wandering person. It should be understood that in actual scenarios, if a person appears near the door of a store with a sideways orientation, it usually means that the person is not clearly heading in the direction of entering the store, but may be considering whether to enter the store, check the window, or just pass by. This behavior pattern indicates that they have not yet made a decision whether to become a real customer, and are therefore classified as a "wandering person."

[0092] Exemplarily, in step S6, if the human body orientation recognition result is the back, the human body target data is regarded as a person leaving, and the passenger flow statistics count is reduced by one. It should be understood that in actual scenarios, if a person appears near the door of a store with the back facing, it usually means that the person is or is about to leave the store. When people leave a store, they naturally turn their backs to the entrance, so the back facing is a strong signal that the person is no longer a customer in the store, but is walking out of the store.

[0093] Exemplarily, in the steps S7 and S8, each human target data in the set of human target data is subjected to object re-identification to obtain a set of object re-identification results, and in response to the object re-identification result being the same person entering the store, the human target data is not repeatedly counted. It should be understood that the object re-identification of each human target data in the set of human target data to obtain a set of object re-identification results is to ensure that the same person is not repeatedly counted during the passenger flow counting process, and that different individuals can be accurately distinguished. In actual application scenarios, customers may enter and exit the store multiple times, or non-customer personnel such as staff and deliverymen may also appear within the monitoring range. If not distinguished, these situations will cause distortion of passenger flow statistics and affect the decisions made by merchants based on this data. Through object re-identification technology, the system can identify the same individual at different time points and camera angles, thereby avoiding the problem of repeated counting, and at the same time effectively eliminate data interference from non-customer personnel, thereby improving the accuracy and reliability of passenger flow statistics.

[0094] In one embodiment, object re-identification is performed on each human target data in the set of human target data to obtain a set of object re-identification results, including: first, each detected human target feature needs to be extracted from the surveillance image, including but not limited to significant features such as appearance, posture, and clothing. Then, a powerful feature encoder is trained using a deep learning model, which can convert complex and changeable human targets into compact and discriminative feature vectors. Next, the system will compare all human target feature vectors detected in the current frame with the previously recorded historical feature library to find the match with the highest similarity. If a match is found, it is considered that the same person has appeared again; if no sufficiently similar result is found, it is regarded as a newly appeared individual and added to the historical feature library.

[0095] In summary, the passenger flow statistics analysis method based on ReID assistance according to the embodiment of the present application is explained, which performs human target detection on the collected door scene monitoring images, and then uses the image analysis and recognition algorithm based on artificial intelligence and machine vision to analyze each human target data, so as to capture the significant feature representation of the human target in each human target data, which is helpful for human orientation discrimination to obtain human orientation recognition results, including front, back and side. In this way, the behavior pattern recognition of each human target object at the door of the store can be performed, so as to identify and detect its orientation, which is helpful for subsequent passenger flow statistics analysis and provides strong support for business decisions.

[0096] Figure 7FIG. 1 is a schematic block diagram of a passenger flow statistics analysis system based on reid assistance according to an embodiment of the present application. Figure 7 As shown, the passenger flow statistics analysis system 100 based on ReID assistance includes: a door scene monitoring image acquisition module 110, which is used to acquire the door scene monitoring image collected by the camera deployed at the door of the store; a human target detection module 120, which is used to perform human target detection on the door scene monitoring image to obtain a set of human target data; a human orientation recognition module 130, which is used to use the trained ReID auxiliary model to perform human orientation recognition on each human target data in the set of human target data to obtain a set of human orientation recognition results; a store entry personnel statistics module 140, which is used to regard the human target data as a store entry personnel and increase the passenger flow statistics count by one if the human orientation recognition result is front; a wandering personnel determination module 150, which is used to regard the human target data as a wandering personnel if the human orientation recognition result is side; a leaving personnel statistics module 160, which is used to regard the human target data as a leaving personnel and decrease the passenger flow statistics count by one if the human orientation recognition result is back.

[0097] Here, those skilled in the art can understand that the specific operations of the various modules and units in the above-mentioned passenger flow statistics analysis system based on reid assistance have been referred to above. Figures 1 to 6 It has been introduced in detail in the description of the re-id-assisted passenger flow statistical analysis method, and therefore, its repeated description will be omitted.

[0098] An embodiment of the present application further provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the methods in the above embodiments of the present application.

[0099] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer implements the methods in the above embodiments of the present application.

[0100] An embodiment of the present application also provides a chip, including a circuit, for executing the methods in the above embodiments of the present application.

[0101] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0102] The prefixes such as "first" and "second" used in the embodiments of the present application are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers used to distinguish description objects in the embodiments of the present application does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary limitation.

[0103] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0104] In the various embodiments of the present application, unless otherwise specified or logically conflicting, the terms and / or descriptions between the various embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A passenger flow statistical analysis method based on reid assistance, characterized in that: include: Obtaining door scene monitoring images collected by cameras deployed at the store entrance; Performing human target detection on the door scene monitoring image to obtain a set of human target data; Using the trained ReID auxiliary model, human body orientation recognition is performed on each human body target data in the set of human body target data to obtain a set of human body orientation recognition results; If the human body orientation recognition result is positive, the human body target data is regarded as a person entering the store and the customer flow statistics count is increased by one; If the human body orientation recognition result is sideways, the human body target data is regarded as a wandering person; If the human body orientation recognition result is the back, the human body target data is regarded as a person leaving, and the passenger flow statistics count is reduced by one.

2. The passenger flow statistical analysis method based on reid assistance according to claim 1 is characterized in that: Also includes the steps: Performing object re-identification on each human target data in the set of human target data to obtain a set of object re-identification results; In response to the object re-identification result being the same person entering the store, the human target data is not repeatedly counted.

3. The passenger flow statistical analysis method based on reid assistance according to claim 2 is characterized in that: Using the trained reid auxiliary model to perform human body orientation recognition on each human body target data in the set of human body target data to obtain a set of human body orientation recognition results, including: Inputting the human target data into a human multi-scale feature extractor based on a dynamic convolutional labeling model to obtain a human target multi-scale image feature coding map; Performing feature selection on the multi-scale image feature coding map of the human target to obtain a significant feature coding map of the human target image; The human body orientation is determined based on the significant feature coding map of the human body target image to obtain the human body orientation recognition result.

4. The passenger flow statistical analysis method based on reid assistance according to claim 3 is characterized in that: Inputting the human target data into a human multi-scale feature extractor based on a dynamic convolutional labeling model to obtain a human target multi-scale image feature coding map, including: Performing local image block segmentation on the human target data to obtain a set of human target local image blocks; Input each human target local image block in the set of human target local image blocks into a human target local semantic feature extractor based on CNN to obtain a set of human target local feature maps; The set of local feature maps of the human target is spliced ​​into a global representation feature map of the human target and then input into a global association encoder of human target features based on a Transformer model to obtain a multi-scale image feature coding map of the human target.

5. The passenger flow statistical analysis method based on reid assistance according to claim 4 is characterized in that: Performing feature selection on the multi-scale image feature coding map of the human target to obtain a significant feature coding map of the human target image, including: Performing feature decomposition on the human target multi-scale image feature coding map to obtain a set of local feature vectors of the human target multi-scale image semantic coding; Inputting each human target multi-scale image semantic coding local feature vector in the set of human target multi-scale image semantic coding local feature vectors into an importance measurement module to obtain a set of human target multi-scale image semantic coding local feature importance score values; Based on the set of importance score values ​​of local features of the multi-scale image semantic coding of the human target, feature selection and feature shape reshaping are performed on the set of local feature vectors of the multi-scale image semantic coding of the human target to obtain a significant feature coding map of the human target image.

6. The passenger flow statistical analysis method based on reid assistance according to claim 5 is characterized in that: The method comprises performing feature decomposition on the multi-scale image feature coding map of the human target to obtain a set of local feature vectors of the multi-scale image semantic coding of the human target, including: performing feature decoupling and feature flattening on the multi-scale image feature coding map of the human target to obtain a set of local feature vectors of the multi-scale image semantic coding of the human target.

7. The passenger flow statistical analysis method based on reid assistance according to claim 6 is characterized in that: Based on the set of importance score values ​​of the local features of the multi-scale image semantic coding of the human target, feature selection and feature shape reshaping are performed on the set of local feature vectors of the multi-scale image semantic coding of the human target to obtain a significant feature coding map of the human target image, including: Based on the set of importance score values ​​of the local features of the multi-scale image semantic coding of the human target, the set of the local feature vectors of the multi-scale image semantic coding of the human target is arranged in descending order to obtain a descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target; Based on the neighborhood features of each human target multi-scale image semantic coding local feature vector in the descending sequence of the human target multi-scale image semantic coding local feature vector, calculating the feature neighborhood activity of each human target multi-scale image semantic coding local feature vector to obtain a sequence of feature neighborhood activity; Based on the sequence of the feature neighborhood activity, feature selection is performed on the descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target to obtain a descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target after selection; The descending sequence of the semantically encoded local feature vectors of the selected human target multi-scale image is reshaped to obtain a significant feature encoding map of the human target image.

8. The passenger flow statistical analysis method based on reid assistance according to claim 7 is characterized in that: Based on the sequence of the feature neighborhood activity, feature selection is performed on the descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target to obtain a descending sequence of the local feature vectors of the multi-scale image semantic coding of the human target after selection, including: Extracting the feature neighborhood activity corresponding to the first human target multi-scale image semantic encoding local feature vector from the sequence of feature neighborhood activity to obtain a first feature neighborhood activity; The first feature neighborhood activity is compared with a predetermined threshold, and in response to the first feature neighborhood activity being greater than or equal to the predetermined threshold, the first human target multi-scale image semantically encoded local feature vector is retained, otherwise the first human target multi-scale image semantically encoded local feature vector is deleted.

9. The passenger flow statistical analysis method based on reid assistance according to claim 8, characterized in that: Based on the significant feature coding map of the human target image, human body orientation is discriminated to obtain the human body orientation recognition result, including: inputting the significant feature coding map of the human target image into a human body orientation discriminator based on a classifier to obtain the human body orientation recognition result.

10. A passenger flow statistics analysis system based on reid assistance, characterized in that: include: The door scene monitoring image acquisition module is used to acquire the door scene monitoring image collected by the camera deployed at the door of the store; A human target detection module, used for performing human target detection on the door scene monitoring image to obtain a set of human target data; A human body orientation recognition module, used for performing human body orientation recognition on each human body target data in the set of human body target data by using the trained ReID auxiliary model to obtain a set of human body orientation recognition results; A store entry personnel statistics module, configured to, if the human body orientation recognition result is positive, regard the human body target data as a store entry person and increase the customer flow statistics count by one; a wandering person determination module, configured to regard the human target data as a wandering person if the human body orientation recognition result is sideways; The leaving person statistics module is used to regard the human target data as a leaving person if the human body direction recognition result is the back, and the passenger flow statistics count is reduced by one.

Images