Passenger flow statistical method and system based on combination of deep learning and monitoring equipment, and electronic equipment

Abstract

The invention discloses a passenger flow statistical method based on combination of deep learning and monitoring equipment. The method comprises the following steps: acquiring real-time monitoring video stream data; extracting pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; recognizing and analyzing pedestrian bbox data sets of all passenger flows in combination with pedestrian ROI image data; performing optimization logic on the pedestrian bbox data set to screen out an optimal frame bbox data set beneficial to passenger flow statistics; initializing a similarity feature library, wherein the similarity feature library comprises a worker feature information storage area and a passenger flow feature information storage area; and performing passenger flow statistical processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistical data. According to the technical scheme provided by the invention, the retail store passenger flow identification efficiency and quality can be improved, and the invention further provides a passenger flow statistical system and electronic equipment for improving the retail store passenger flow identification efficiency and quality based on the combination of deep learning and monitoring equipment, which also have the above technical effects.

Description

Technical Field

[0001] The present invention relates to the field of passenger flow counting technology, and more specifically, to a passenger flow counting method based on the combination of deep learning and monitoring equipment, and also to a passenger flow counting system and electronic equipment based on the combination of deep learning and monitoring equipment. Background Art

[0002] Fast and accurate statistics of retail store traffic is of great significance to enterprise development and resource scheduling. Retail stores are the front line of enterprise product sales and the foundation of enterprise development. By counting store traffic, it can help retail enterprises better understand customer flow, accurately analyze each retail store, and formulate more precise business strategies to enhance competitiveness.

[0003] In the prior art, most retail store customer flow counting solutions use manual customer flow counting, the main drawback of which is that it is difficult to count the customer flow in the station in a timely and accurate manner, and it requires a lot of manpower and financial resources, especially during peak holiday periods.

[0004] Therefore, how to provide a customer flow statistics method based on the combination of deep learning and monitoring equipment, which can improve the efficiency and quality of customer flow identification in retail stores, has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] To solve the above technical problems, the present invention provides a customer flow counting method based on deep learning combined with monitoring equipment, which can improve the efficiency and quality of customer flow identification in retail stores. The present invention also provides a maintenance method, which also has the above technical effects.

[0006] The technical solution provided by the present invention is as follows:

[0007] The present application relates to a technical solution of a passenger flow counting method based on the combination of deep learning and monitoring equipment, the method comprising the following steps: acquiring real-time monitoring video stream data; extracting pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; identifying and analyzing pedestrian bbox data sets of all passenger flows in combination with the pedestrian ROI image data; screening the pedestrian bbox data sets, and outputting an optimal frame bbox data set that is conducive to passenger flow statistics, wherein the optimal frame bbox data in the optimal frame bbox data set is the optimal frame bbox of each pedestrian; initializing a similarity feature library, wherein the similarity feature library comprises a staff feature information storage area and a passenger flow feature information storage area, wherein the staff feature information storage area is used to store staff feature information, and the passenger flow feature information storage area is used to store passenger flow feature information compared to a new passenger flow; performing passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistics data.

[0008] Furthermore, in a preferred embodiment of the present invention, the similarity feature library stores feature information by establishing ID numbers: the feature information stored in the staff feature information storage area is associated with the corresponding staff ID, and the feature information stored in the passenger flow feature information storage area is associated with the corresponding pedestrian ID.

[0009] Further, in a preferred embodiment of the present invention, the “performing passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set” specifically includes the following steps: the optimal frame bbox data set includes optimal frame bbox data updated in real time; combining the staff feature information in the staff feature information storage area, performing a first similarity screening on the optimal frame bbox data set:

[0010] If the human feature data in a certain optimal frame bbox data is similar to the feature information of the staff, the analysis and statistics of the optimal frame bbox data will be skipped and the analysis and statistics of the next optimal frame bbox data will be entered;

[0011] If the human feature data in a certain optimal frame bbox data is not similar to the staff feature information, a second similarity screening is further performed;

[0012] The second similarity screening: if the human body feature data of the optimal frame bbox data is similar to the passenger flow feature information of the current history, it is judged to be the same passenger flow data; the passenger flow feature information of the current history is updated in combination with the human body feature data of the optimal frame bbox data; if the human body feature data of the optimal frame bbox data is not similar to the passenger flow feature information of the current history, it is judged to be new passenger flow data, a new passenger flow ID is generated for the new passenger flow, the new passenger flow ID and the human body feature data of the optimal frame bbox data are stored in the passenger flow feature information storage area as the passenger flow feature information of the new passenger flow, and are included in the passenger flow statistics.

[0013] Furthermore, in a preferred embodiment of the present invention, the "optimal logical screening of the pedestrian bbox data set to obtain the optimal frame bbox data set that is conducive to passenger flow statistics" specifically includes the following steps: performing high-quality bbox screening processing on the pedestrian bbox data set to obtain a first screened passenger flow bbox data set that does not contain other pedestrian background features; performing facial feature recognition on the first screened passenger flow bbox data set to filter out bbox data whose facial feature information cannot be identified or whose facial feature information is unclear to obtain a second screened passenger flow bbox data set; the facial feature information is nose feature information; performing optimal frame bbox screening on the second screened passenger flow bbox data set to filter out the bbox data that is most conducive to facial feature recognition to obtain the optimal frame bbox data set.

[0014] Furthermore, in a preferred embodiment of the present invention, the “optimizing logic to screen out the optimal frame bbox data set that is beneficial to passenger flow statistics from the pedestrian bbox data set” specifically includes the following steps: before outputting the optimal frame bbox data set, desensitizing the pedestrian bbox data of the optimal frame bbox data set: filling the head area in the bbox with mosaics.

[0015] Further, in a preferred embodiment of the present invention, the “screening the second screened passenger flow bbox data set for the optimal frame bbox, screening out the bbox data that is most conducive to facial feature recognition, and obtaining the optimal frame bbox data set” specifically includes the following steps: by calculating and analyzing the numerical relationship between the nose coordinates and the head center point coordinates, the degree to which the pedestrian faces the camera is judged, and the bbox data that is identified as meeting the front requirements is used as the optimal frame bbox data; the calculation and analysis of the numerical relationship between the nose coordinates and the head center point coordinates is specifically to calculate and analyze the Socre value, and the formula is as follows:

[0016]

[0017] std w =(x2-x1) / s w Formula (3)

[0018] std h =(y2-y1) / s h Formula (4)

[0019]

[0020] Score=Score x *Score y Formula (7)

[0021] Among them: (x1, y1), (x2, y2) are the position coordinates of the upper left corner and lower right corner of the head output by bbox, (x3, y3) are the position coordinates of the nose of the head output by bbox; w 、s h represents the head bounding box hyperparameter; α is the optimal face frame hyperparameter; Socre represents the score of the bbox as the optimal frame bbox. The optimal frame bbox can be screened out by setting the score threshold.

[0022] Furthermore, in a preferred embodiment of the present invention, the step of "performing facial feature recognition on the first filtered passenger flow bbox data set, filtering out bbox data for which facial feature information cannot be recognized or facial feature information is unclear, and obtaining a second filtered passenger flow bbox data set" specifically includes the following steps: defining bbox data for which accurate facial feature information can be recognized as data entering a passenger flow statistical area; defining bbox data for which facial feature information cannot be recognized or facial feature information is unclear as data leaving a passenger flow statistical area; filtering out data leaving the passenger flow statistical area, and aggregating the data entering the passenger flow statistical area into a second filtered passenger flow bbox data set.

[0023] Furthermore, in a preferred embodiment of the present invention, the step of "performing high-quality bbox screening processing on the pedestrian bbox data set to obtain a first screened passenger flow bbox data set that does not contain other pedestrian background features" specifically includes the following steps: identifying the boundary coordinates of all bbox data in the pedestrian bbox data set; performing cross-judgment on two adjacent bboxes along a fixed direction based on the boundary coordinates of all bbox data; deleting any two cross-overlapping bbox data, and retaining the non-cross-overlapping bbox data after the cross-judgment, to obtain a first screened passenger flow bbox data set.

[0024] In addition, the present application also relates to a technical solution of a passenger flow counting system based on the combination of deep learning and monitoring equipment. The system is used to execute the above-mentioned passenger flow counting method based on the combination of deep learning and monitoring equipment; the system includes: a first acquisition module, the first acquisition module is used to acquire real-time monitoring video stream data; a first data processing module, the first data processing module is used to extract pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; a second data processing module, the second data processing module is used to combine the pedestrian ROI image data to identify and analyze the pedestrian bbox data set of all passenger flows; a third data processing module, the third data processing module is used to perform optimal logic selection on the pedestrian bbox data set to obtain the optimal frame bbox data set that is conducive to passenger flow statistics; a data storage module, the data storage module includes a similarity feature library; a fourth data processing module, the fourth data processing module is used to perform passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistics data. The technical solution of a passenger flow counting system based on the combination of deep learning and monitoring equipment provided in the present application also has the technical effect of the above-mentioned technical solution.

[0025] In addition, the present application also provides a technical solution for an electronic device. The solution includes: a computer program, the computer program is used to execute the above-mentioned passenger flow counting method based on the combination of deep learning and monitoring equipment; a memory, the memory is used to store the computer program; and a processor is used to execute the computer program. The technical solution of the electronic device provided by the present application also has the technical effect of the above-mentioned technical solution.

[0026] Compared with the prior art, the present invention provides a technical solution for a passenger flow counting method based on the combination of deep learning and monitoring equipment. In this technical solution, firstly, the pedestrian ROI image data is obtained for the designated statistical area in the real-time acquired monitoring video stream data; then the pedestrian ROI image data is analyzed to obtain the pedestrian bbox data set; then the pedestrian bbox data set is screened to obtain the optimal frame bbox data set; finally, the passenger flow is counted by combining the similarity judgment method; in the above experiment, the artificial intelligence recognition method is applied to improve the efficiency and accuracy of passenger flow statistics. Compared with the prior art, the technical solution of the passenger flow counting method based on the combination of deep learning and monitoring equipment provided by the present application can improve the efficiency and quality of passenger flow identification in retail stores. The present invention also provides a passenger flow counting system and electronic equipment based on the combination of deep learning and monitoring equipment to improve the efficiency and quality of passenger flow identification in retail stores, which also has the above technical effects. Compared with the prior art, the technical advantages of the present invention include: a single model is used to output the human body position frame and perform desensitization processing during pedestrian detection and recognition, which reduces the time spent on hardware identification and greatly improves real-time statistics compared with the past; the optimal frame logic screening logic is designed to improve the accuracy of passenger flow statistics, reducing the statistical number higher than the actual number due to confusion in passenger flow identification; it can be extended and not limited to a single store, and the model parameters can be modified and the model can be migrated according to the specific store, so that it can be quickly applied and implemented. Taking into account the actual application scenarios, no complex model is used to achieve reliable results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A flow chart of the main steps of a passenger flow counting method based on deep learning combined with monitoring equipment provided in an embodiment of the present invention;

[0029] Figure 2 This is a flow chart of an embodiment of a passenger flow statistics method based on the combination of deep learning and monitoring equipment involved in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that when an element is referred to as being "fixed on" or "set on" another element, it can be directly on the other element or indirectly set on the other element; when an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0032] It should be understood that the terms "length", "width", "up", "down", "front", "back", "first", "second", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0033] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" and "several" is two or more, unless otherwise clearly and specifically defined.

[0034] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0035] like Figure 1 to Figure 2As shown, a passenger flow counting method based on the combination of deep learning and monitoring equipment provided by an embodiment of the present invention includes the following steps: acquiring real-time monitoring video stream data; extracting pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; identifying and analyzing pedestrian bbox data sets of all passenger flows in combination with the pedestrian ROI image data; screening the pedestrian bbox data set, and outputting an optimal frame bbox data set that is conducive to passenger flow statistics, wherein the optimal frame bbox data in the optimal frame bbox data set is the optimal frame bbox of each pedestrian; initializing a similarity feature library, wherein the similarity feature library includes a staff feature information storage area and a passenger flow feature information storage area, wherein the staff feature information storage area is used to store staff feature information, and the passenger flow feature information storage area is used to store passenger flow feature information compared to a new passenger flow; performing passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistics data.

[0036] The embodiment of the present invention provides a technical solution for a passenger flow counting method based on the combination of deep learning and monitoring equipment. In this technical solution, firstly, the pedestrian ROI image data is obtained for the designated statistical area in the real-time monitoring video stream data; then the pedestrian ROI image data is analyzed to obtain the pedestrian bbox data set; then the pedestrian bbox data set is screened to obtain the optimal frame bbox data set; finally, the passenger flow is counted by combining the similarity judgment method; in the above experiment, the artificial intelligence recognition method is applied to improve the efficiency and accuracy of passenger flow statistics. Compared with the prior art, the technical solution of the passenger flow counting method based on the combination of deep learning and monitoring equipment provided by the present application can improve the efficiency and quality of passenger flow identification in retail stores. The present invention also provides a passenger flow counting system and electronic equipment based on the combination of deep learning and monitoring equipment to improve the efficiency and quality of passenger flow identification in retail stores, which also has the above technical effects.

[0037] It should be noted that the optimal frame bbox data in the optimal frame bbox data set is the optimal frame bbox of each pedestrian. Since the optimal frame bbox represents the optimal bbox of each pedestrian, the passenger flow counted thereby will be more accurate.

[0038] Specifically, in an embodiment of the present invention, the similarity feature library stores feature information by establishing ID numbers: the feature information stored in the staff feature information storage area is associated with the corresponding staff ID, and the feature information stored in the passenger flow feature information storage area is associated with the corresponding pedestrian ID.

[0039] It should be noted that associating and managing all customer flow data through ID identities is conducive to the design of marketing activities in retail stores and provides data support and reference.

[0040] Specifically, in an embodiment of the present invention, the “performing passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set” specifically includes the following steps: the optimal frame bbox data set includes optimal frame bbox data updated in real time; combining the staff feature information in the staff feature information storage area, performing a first similarity screening on the optimal frame bbox data set:

[0041] If the human feature data in a certain optimal frame bbox data is similar to the feature information of the staff, the analysis and statistics of the optimal frame bbox data will be skipped and the analysis and statistics of the next optimal frame bbox data will be entered;

[0042] If the human feature data in a certain optimal frame bbox data is not similar to the staff feature information, a second similarity screening is further performed;

[0043] The second similarity screening: if the human body feature data of the optimal frame bbox data is similar to the passenger flow feature information of the current history, it is judged to be the same passenger flow data; the passenger flow feature information of the current history is updated in combination with the human body feature data of the optimal frame bbox data; if the human body feature data of the optimal frame bbox data is not similar to the passenger flow feature information of the current history, it is judged to be new passenger flow data, a new passenger flow ID is generated for the new passenger flow, the new passenger flow ID and the human body feature data of the optimal frame bbox data are stored in the passenger flow feature information storage area as the passenger flow feature information of the new passenger flow, and are included in the passenger flow statistics.

[0044] It should be noted that in the process of screening the optimal frame bbox data, the erroneous or poor-quality bbox data are deleted through multiple screening to improve the accuracy of the overall solution.

[0045] Specifically, in an embodiment of the present invention, the "optimal logical screening of the pedestrian bbox data set to obtain the optimal frame bbox data set that is conducive to passenger flow statistics" specifically includes the following steps: performing high-quality bbox screening processing on the pedestrian bbox data set to obtain a first screened passenger flow bbox data set that does not contain other pedestrian background features; performing facial feature recognition on the first screened passenger flow bbox data set to filter out bbox data whose facial feature information cannot be identified or whose facial feature information is unclear to obtain a second screened passenger flow bbox data set; the facial feature information is nose feature information; performing optimal frame bbox screening on the second screened passenger flow bbox data set to filter out the bbox data that is most conducive to facial feature recognition to obtain the optimal frame bbox data set.

[0046] Specifically, in an embodiment of the present invention, the "optimal logic screening of the pedestrian bbox dataset to obtain the optimal frame bbox dataset that is beneficial to passenger flow statistics" specifically also includes the following steps: before the optimal frame bbox dataset is output, the pedestrian bbox data is desensitized on the optimal frame bbox dataset: the head area in the bbox is filled with mosaics.

[0047] Specifically, in an embodiment of the present invention, the "screening the second screened passenger flow bbox data set for the optimal frame bbox, screening out the bbox data that is most conducive to facial feature recognition, and obtaining the optimal frame bbox data set" specifically includes the following steps: by calculating and analyzing the numerical relationship between the nose coordinates and the head center point coordinates, the degree to which the pedestrian faces the camera is judged, and the bbox data that is identified as meeting the front requirement is used as the optimal frame bbox data; the calculation and analysis of the numerical relationship between the nose coordinates and the head center point coordinates is specifically to calculate and analyze the Socre value, and the formula is as follows:

[0048]

[0049] std w =(x2-x1) / s w Formula (3)

[0050] std h =(y2-y1) / s h Formula (4)

[0051]

[0052] Score=Score x *Score y Formula (7)

[0053] Among them: (x1, y1), (x2, y2) are the position coordinates of the upper left corner and lower right corner of the head output by bbox, (x3, y3) are the position coordinates of the nose of the head output by bbox; w 、s h represents the head bounding box hyperparameter; α is the optimal face frame hyperparameter; Socre represents the score of the bbox as the optimal frame bbox. The optimal frame bbox can be screened out by setting the score threshold.

[0054] Specifically, in an embodiment of the present invention, the "performing facial feature recognition on the first filtered passenger flow bbox data set, filtering out bbox data for which facial feature information cannot be recognized or facial feature information is unclear, and obtaining a second filtered passenger flow bbox data set" specifically includes the following steps: defining bbox data that can accurately recognize facial feature information as data entering the passenger flow statistical area; defining bbox data for which facial feature information cannot be recognized or facial feature information is unclear as data leaving the passenger flow statistical area; filtering out the data leaving the passenger flow statistical area, and aggregating the data entering the passenger flow statistical area into the second filtered passenger flow bbox data set.

[0055] Specifically, in an embodiment of the present invention, the "performing high-quality bbox screening processing on the pedestrian bbox data set to obtain a first screened passenger flow bbox data set that does not contain other pedestrian background features" specifically includes the following steps: identifying the boundary coordinates of all bbox data in the pedestrian bbox data set; performing intersection judgment on two adjacent bboxes along a fixed direction according to the boundary coordinates of all bbox data; deleting any two intersecting and overlapping bbox data, and retaining the non-intersecting and overlapping bbox data after the intersection judgment, to obtain the first screened passenger flow bbox data set.

[0056] In addition, the present application also relates to a technical solution of a passenger flow counting system based on the combination of deep learning and monitoring equipment. The system is used to execute the above-mentioned passenger flow counting method based on the combination of deep learning and monitoring equipment; the system includes: a first acquisition module, the first acquisition module is used to acquire real-time monitoring video stream data; a first data processing module, the first data processing module is used to extract pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; a second data processing module, the second data processing module is used to combine the pedestrian ROI image data to identify and analyze the pedestrian bbox data set of all passenger flows; a third data processing module, the third data processing module is used to perform optimal logic selection on the pedestrian bbox data set to obtain the optimal frame bbox data set that is conducive to passenger flow statistics; a data storage module, the data storage module includes a similarity feature library; a fourth data processing module, the fourth data processing module is used to perform passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistics data. The technical solution of a passenger flow counting system based on the combination of deep learning and monitoring equipment provided in the present application also has the technical effect of the above-mentioned technical solution.

[0057] In addition, the present application also provides a technical solution for an electronic device. The solution includes: a computer program, the computer program is used to execute the above-mentioned passenger flow counting method based on the combination of deep learning and monitoring equipment; a memory, the memory is used to store the computer program; and a processor is used to execute the computer program. The technical solution of the electronic device provided by the present application also has the technical effect of the above-mentioned technical solution.

[0058] It should be noted that the technical solution provided by this application is intended to meet the urgent needs of modern society for accurate and efficient data management, especially in the fields of commercial operations, urban planning and public safety. Its core purpose can be summarized as follows: First, to improve data accuracy and real-time performance. Traditional manual counting or simple electronic equipment statistical methods are not only time-consuming and labor-intensive, but also susceptible to human factors and environmental interference, resulting in large data errors. The invention of the new passenger flow statistics solution uses advanced video recognition, Internet of Things and other technologies to achieve accurate capture and real-time transmission of passenger flow data, providing managers with more reliable data support. Secondly, optimize business decisions and operational efficiency. Through in-depth analysis of passenger flow, merchants can accurately grasp customer behavior patterns, such as peak hours, popular areas, etc., so as to scientifically adjust product layout, promotion strategies and business hours, and improve customer satisfaction and store profitability. At the same time, real-time feedback of data also provides the possibility of rapid response in emergency situations, such as timely deployment of manpower and adjustment of inventory.

[0059] In summary, the present invention provides a customer flow counting method based on deep learning and surveillance cameras, which can provide accurate customer flow counting for retail stores, shopping malls, restaurants and other scenarios, reduce manpower and material resources, and improve the quality of customer flow counting.

[0060] It should be noted that the specific implementation plan provided in this application is as follows:

[0061] A method for counting customer flow in retail stores based on deep learning and monitoring equipment. The technical problem to be solved is achieved through the following technical solutions:

[0062] Step 1: Install surveillance equipment in retail stores to obtain real-time surveillance video streams of the stores, and set up high-density customer flow counting areas to accurately capture ROI images of all pedestrians entering and leaving the stores.

[0063] Step 2: Based on the obtained pedestrian ROI image, use the target detection model (yolov8) to identify all passenger flow bboxes in the image. The main steps include two steps:

[0064] 1) After the ROI image is input into the target detection model, all pedestrian boxes (bboxes) in the image will be output. Each bbox contains four parts of information: the coordinates of the human body position box, the coordinates of the head position box, and the coordinates of the nose position, and is marked as belonging to passenger flow or non-passenger flow (couriers, takeaways, etc.).

[0065] 2) The purpose of passenger flow statistics is to identify customers, so bboxes identified as non-passenger flow will be filtered out. For bboxes identified as passenger flow, a high confidence threshold is set to ensure that the bbox is a real passenger flow to prevent subsequent disruption of the statistical system. Finally, the retained passenger flow bboxes are collected into the bboxes temporary cache area.

[0066] Step 3: Based on the collected bboxes, the optimal frame bbox that is most conducive to passenger flow statistics needs to be screened out through the optimization logic, which mainly includes four steps:

[0067] 1) High-quality bbox screening: The purpose of this step is to screen out pedestrian bboxes that do not contain other pedestrian backgrounds.

[0068] First, take out all the bboxes in the temporary buffer. Since the temporary buffer stores all the pedestrian bboxes at a certain moment,

[0069] First, sort all bboxes according to the horizontal coordinate of the left boundary, and set all bbox labels to 1 (0 means not to keep, 1 means to keep);

[0070] Then, traverse from left to right and take out two bboxes in turn by moving forward one bbox at a time: bbox1 (left) and bbox2 (right). If the left boundary of bbox2 is larger than the right boundary of bbox1, the two bboxes do not have an intersection area, and enter the next traversal; if the left boundary of bbox2 is smaller than the right boundary of bbox1, the two bboxes have an intersection area, and the labels of the two bboxes are set to 0.

[0071] Finally, only the bbox marked as 1 is retained.

[0072] 2) Bbox screening for entering and exiting the store: In the process of counting customer flow, it is considered that pedestrians entering the store need to be counted, while pedestrians leaving the store have already been counted when entering, so there is no need to count them again.

[0073] For each bbox, we believe that the bbox with low nose confidence or no nose information is out of the store. This type of bbox has been counted, so we filter it out. For the bbox with high nose confidence, it belongs to entering the store. This type of bbox is very likely to be a new customer flow that needs to be counted, so we retain it.

[0074] 3) Optimal frame bbox screening: The purpose of this step is to screen out the bbox that is most conducive to feature extraction. From the perspective of ergonomics, the most favorable posture for capturing pedestrian features should be that the monitoring angle of the pedestrian front monitoring device is as close to 90 degrees as possible, that is, for the human body bbox, the nose coordinate will be at the center of the head. Therefore, the following formula is designed to measure whether the face is facing forward or not to screen out the optimal frame bbox.

[0075] The formula will output a value between 0 and 1, which represents the degree to which the pedestrian faces the camera. By setting an empirical threshold, the optimal frame bbox can be screened out. The overall calculation formula is as follows:

[0076]

[0077] std w =(x2-x1) / s w

[0078] std h =(y2-y1) / s h

[0079]

[0080] Score=Score x *Score y

[0081] Among them: (x1, y1) and (x2, y2) represent the position coordinates of the upper left corner and lower right corner of the head output by bbox; s w 、s h represents the head bounding box hyperparameter; α is the optimal face frame hyperparameter; Socre represents the score of the bbox as the optimal frame bbox. The optimal frame bbox can be screened out by setting the score threshold.

[0082] 4) Pedestrian bbox desensitization: The purpose of this step is to ensure the privacy and security of customers to the greatest extent. For each bbox, we fill the head position box with mosaics. The desensitized bbox will be saved as the passenger flow bbox and enter the next stage.

[0083] Step 4: Initialize the similarity feature library. The feature library contains two accessible areas, which are explained in detail below:

[0084] 1) Area 1: This area is used to store staff feature information. When the machine is turned on for the first time, each staff member must face the visual sensor to record image information while rotating 45 degrees to the left and right to record image information. This area is used to prevent staff from being mistakenly identified as passengers.

[0085] 2) Area 2: This area is used to store passenger flow feature information. When the device is started daily, this area is empty. As passenger flow continues to enter, we store different passenger flow features in this area in a stacked manner.

[0086] When a passenger flow feature visits the area, we perform feature comparison from the top of the stack and set the maximum number of comparisons.

[0087] When the maximum number of comparisons is exceeded, it is considered to be a new passenger flow and its features are stored at the top;

[0088] When the best similar feature is found before the maximum number of comparisons, we consider it as the historical passenger flow, and take the average of the two features to update the feature vector and store it at the top.

[0089] Step 5: For the desensitized bbox that is screened as the optimal frame, it is sent to the ReID model to extract the feature vector of the customer flow. Since the store clerks always appear in the surveillance images, for each bbox, it is first compared with the region 1 in the feature library for similarity to determine whether it belongs to the staff. When it is identified as a staff member, it will exit the statistical task of the bbox; if it does not match with region 1, it will be compared with the historical customer flow features in region 2 in the feature library for feature similarity. When the similarity exceeds the set similarity threshold, its ID is marked as the same ID as the historical customer flow. When it cannot be matched with the historical customer flow, we store its features as a new customer flow ID, and the customer flow increases by one.

[0090] It should be added that Figure 2 As shown, in a specific embodiment of the present application, surveillance video data is acquired in real time, and an analysis and statistical area is set in the surveillance video data; the optimal frame bbox data of the surveillance video is analyzed in combination with the yolov8 model, and firstly, it is determined whether it is passenger flow. If so, the body coordinates, head coordinates and nose coordinates in the bbox data are further determined, and the above data are all stored in bboxes; then, the selection is performed through the optimization logic, and the front face images are selected to enter the passenger flow statistical processing link, and the non-front face images are also discarded. In the passenger flow statistical processing link, the ReID model is used to judge the similarity based on the body features, and the data that is not similar to the historical data is recorded as New ID, and the total passenger flow is added by 1. If it is determined to be similar, the database data is updated. The above-mentioned technical solution realizes the passenger flow statistical function combined with artificial intelligence.

[0091] It should be noted that customer flow statistics technology is an advanced intelligent statistical technology suitable for scenarios such as shopping malls, retail stores, and restaurants. It can help companies achieve data-driven decision-making and management, improve operational efficiency and service quality, and is of great significance to the digital transformation of enterprises.

[0092] At present, the customer flow statistics of most customer flow counting solutions on the market are much higher than the actual customer flow, which is not conducive to formulating resource scheduling by analyzing customer flow data. Being able to accurately count the customer flow of retail stores is of great significance to both stores and consumers.

[0093] This paper mainly uses surveillance cameras to obtain store surveillance images, combines deep learning algorithm models, and uses custom logic to filter the optimal frame bbox to achieve accurate customer flow statistics in stores.

[0094] Specific approach First, use surveillance cameras to obtain store images, and set up high-density pedestrian entry and exit areas to maximize the guarantee that only customers with the motivation to enter and exit the store are counted. Use yolov8 to detect and identify pedestrians, and output the coordinates of the pedestrian's body, head, and nose in the image. For the bbox identified as passenger flow, it is sent to the next step. For the bbox identified as non-passenger flow, we filter it and record it. Secondly, the optimal frame bbox is selected through logical screening, and it is sent to the ReID model to extract the feature vector, and the similarity is calculated with the feature vectors of the two regions in the feature library. When it has a high similarity with the first feature region, it is identified as a staff member; when the similarity value with the second feature region is high, it is marked as a historical passenger flow. When the similarity value with the second feature region is low, it is marked as a new passenger flow, and its feature vector is pinned to the top of the feature library. The results of the field experiment show that the accuracy rate can reach more than 90% when this method is used to count the passenger flow of retail stores.

[0095] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A passenger flow statistics method based on deep learning combined with monitoring equipment, characterized in that: The method comprises the following steps: Obtain real-time monitoring video stream data; Extract pedestrian ROI image data of a designated statistical area from the real-time monitoring video stream data; Combined with the pedestrian ROI image data, identify and analyze the pedestrian bbox data set of all passenger flows; The pedestrian bbox data set is screened to output an optimal frame bbox data set that is beneficial to passenger flow statistics, wherein the optimal frame bbox data in the optimal frame bbox data set is the optimal frame bbox of each pedestrian; Initialize the similarity feature library; The similarity feature library includes: a staff feature information storage area and a passenger flow feature information storage area; The staff characteristic information storage area is used to store staff characteristic information; The passenger flow characteristic information storage area is used to store passenger flow characteristic information of new passenger flow; The optimal frame bbox data set is subjected to passenger flow statistics processing based on similarity calculation to obtain passenger flow statistics data.

2. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 1 is characterized in that: The similarity feature library stores feature information by setting up ID numbers: the feature information stored in the staff feature information storage area is associated with the corresponding staff ID, and the feature information stored in the passenger flow feature information storage area is associated with the corresponding pedestrian ID.

3. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 2 is characterized in that: The “performing passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set” specifically includes the following steps: The optimal frame bbox data set includes optimal frame bbox data updated in real time; Combined with the staff feature information in the staff feature information storage area, the optimal frame bbox data set is subjected to a first similarity screening: If the human feature data in a certain optimal frame bbox data is similar to the feature information of the staff, the analysis and statistics of the optimal frame bbox data will be skipped and the analysis and statistics of the next optimal frame bbox data will be entered; If the human feature data in a certain optimal frame bbox data is not similar to the staff feature information, a second similarity screening is further performed; The second similarity screening is: If the human feature data in the optimal frame bbox data is similar to the passenger flow feature information of the current history, it is determined to be the same passenger flow data; the passenger flow feature information of the current history is updated in combination with the human feature data in the optimal frame bbox data; If the human feature data in the optimal frame bbox data is not similar to the current historical passenger flow feature information, it is judged as new passenger flow data, and a new passenger flow ID is generated for the new passenger flow number. The new passenger flow ID and the human feature data in the optimal frame bbox data are stored in the passenger flow feature information storage area as the passenger flow feature information of the new passenger flow, and are included in the passenger flow statistics.

4. The passenger flow counting method based on deep learning combined with monitoring equipment according to any one of claims 1 to 3, characterized in that: The "selecting the pedestrian bbox data set by optimal logic to select the optimal frame bbox data set that is beneficial to passenger flow statistics" includes the following steps: Performing high-quality bbox screening processing on the pedestrian bbox dataset to obtain a first screened passenger flow bbox dataset that does not contain other pedestrian background features; Performing facial feature recognition on the first screening passenger flow bbox data set, filtering out bbox data for which facial feature information cannot be recognized or facial feature information is unclear, to obtain a second screening passenger flow bbox data set; the facial feature information is nose feature information; The optimal frame bbox is screened for the second filtered passenger flow bbox data set to screen out bbox data that is conducive to facial feature recognition, thereby obtaining the optimal frame bbox data set.

5. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 4 is characterized in that: The "selecting the pedestrian bbox data set by optimal logic to select the optimal frame bbox data set that is beneficial to passenger flow statistics" also includes the following steps: Before the optimal frame bbox data set is output, pedestrian bbox data desensitization processing is performed on the optimal frame bbox data set: the head area in the bbox is filled with mosaics.

6. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 4 is characterized in that: The "performing optimal frame bbox screening on the second screening passenger flow bbox data set, screening out the bbox data that is most conducive to facial feature recognition, and obtaining the optimal frame bbox data set" specifically includes the following steps: By calculating and analyzing the numerical relationship between the coordinates of the nose and the center point of the head, the degree to which the pedestrian faces the camera is determined, and the bbox data that is identified as meeting the frontal requirements is used as the optimal frame bbox data; The calculation and analysis of the numerical relationship between the nose coordinates and the head center coordinates is specifically to calculate and analyze the Socre value, and the formula is as follows: std w = (x2-x1) / s w Formula(3) std h = (y2-y1) / s h Formula(4) Score=Score x *Score y Formula(7) Among them: (x1, y1), (x2, y2) are the position coordinates of the upper left corner and lower right corner of the head output by bbox, (x3, y3) are the position coordinates of the nose of the head output by bbox; w 、s h represents the head bounding box hyperparameter; α is the optimal face frame hyperparameter; Socre represents the score of the bbox as the optimal frame bbox. The optimal frame bbox can be screened out by setting the score threshold.

7. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 4 is characterized in that: The "performing facial feature recognition on the first screening passenger flow bbox data set, screening out bbox data whose facial feature information cannot be recognized or whose facial feature information is unclear, and obtaining a second screening passenger flow bbox data set" specifically includes the following steps: The bbox data that can accurately identify facial feature information is defined as the data entering the passenger flow statistics area; The bbox data whose facial feature information cannot be identified or is unclear is defined as data leaving the passenger flow statistics area; The data leaving the passenger flow statistics area is screened out, and the data entering the passenger flow statistics area is aggregated into a second screening passenger flow bbox data set.

8. The passenger flow counting method based on deep learning combined with monitoring equipment according to claim 4 is characterized in that: The "performing high-quality Bbox screening processing on the pedestrian Bbox dataset to obtain a first screened passenger flow Bbox dataset that does not contain other pedestrian background features" specifically includes the following steps: Identify the boundary coordinates of all bbox data in the pedestrian bbox dataset; According to the boundary coordinates of all bbox data, cross judgment is performed on two adjacent bboxes along a fixed direction; Delete any two overlapping bbox data, retain the non-overlapping bbox data after the intersection judgment, and obtain the first filtered passenger flow bbox data set.

9. A passenger flow counting system based on deep learning combined with monitoring equipment, characterized in that: The method for counting passenger flow based on deep learning combined with monitoring equipment according to any one of claims 1 to 8 comprises: A first acquisition module, the first acquisition module is used to acquire real-time monitoring video stream data; A first data processing module, the first data processing module is used to extract pedestrian ROI image data of a specified statistical area from the real-time monitoring video stream data; A second data processing module, the second data processing module is used to identify and analyze the pedestrian bbox data set of all passenger flows in combination with the pedestrian ROI image data; A third data processing module, the third data processing module is used to perform a selection logic on the pedestrian bbox data set to select an optimal frame bbox data set that is beneficial to passenger flow statistics; A data storage module, wherein the data storage module includes a similarity feature library; The fourth data processing module is used to perform passenger flow statistics processing based on similarity calculation on the optimal frame bbox data set to obtain passenger flow statistics data.

10. An electronic device, characterized in that: include: A computer program, the computer program being used to execute the passenger flow counting method based on the combination of deep learning and monitoring equipment as described in any one of claims 1 to 8; A memory, the memory being used to store the computer program; A processor is used to execute the computer program.

Images