Passenger flow statistical method and device, computer device and storage medium

Abstract

The application relates to the technical field of passenger flow statistics, and discloses a passenger flow statistics method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring first video data of a first camera at the entrance of a place to be counted; performing head detection on the first video data to obtain a first target head; tracking the first target head to obtain a first head image; determining the behavior state of the first target head based on the tracking route of the first target head; performing face recognition and feature extraction on the first head image under the condition that the quality of the first head image is qualified, and obtaining the face feature value of the first target head; determining the behavior state information of the first target head based on the behavior state of the first target head; saving the face feature value and the behavior state information of the first target head to a passenger flow information statistics table; and determining the passenger flow information of the place to be counted based on the passenger flow information statistics table. The application improves the accuracy of passenger flow statistics results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of passenger flow statistics, in particular to a passenger flow statistics method and device, computer equipment and a storage medium. BACKGROUND

[0002] Traditional retail and service industries rely on passenger flow to measure sales performance. Through the statistics and analysis of passenger flow, merchants can understand customer behavior, optimize operation strategy, improve service quality, and ultimately achieve higher sales performance. With the progress of artificial intelligence and video image analysis technology, the accuracy and efficiency of passenger flow statistics have been significantly improved, providing a more reliable data basis for business decision-making.

[0003] In related technologies, there are various methods for passenger flow statistics, and the commonly used method is video-based passenger flow statistics. The method based on face detection and tracking is usually used to count passenger flow under the video-based passenger flow statistics method.

[0004] However, the method based on face detection and tracking does not take into account the situation that the face cannot be recognized and tracked due to the turning of the head or the lowering of the head during tracking, resulting in low accuracy of the obtained passenger flow statistics results. SUMMARY

[0005] Therefore, the present application provides a passenger flow statistics method to solve the problem of low accuracy of the obtained passenger flow statistics results due to the method based on face detection and tracking not taking into account the situation that the face cannot be recognized and tracked due to the turning of the head or the lowering of the head during tracking in related technologies.

[0006] In a first aspect, the present application provides a passenger flow statistics method, which comprises:

[0007] obtaining first video data of a first camera at the entrance of a place to be counted;

[0008] performing head detection on the first video data to obtain a first target head;

[0009] tracking the first target head to obtain a first head image of the first target head;

[0010] determining the behavior state of the first target head based on the tracking route of the first target head;

[0011] performing face recognition and feature extraction on the first head image to obtain the face feature value of the first target head if the first head image is qualified;

[0012] determining the behavior state information of the first target head based on the behavior state of the first target head;

[0013] saving the facial feature value of the first target head and the behavior state information to a passenger flow information statistical table;

[0014] determining passenger flow information of the place to be counted based on the passenger flow information statistical table.

[0015] The passenger flow statistical method provided in the embodiment prevents the situation of face unrecognizable and tracking failure caused by head turning or head lowering by performing head detection on the first video data of the first camera at the entrance of the place to be counted to obtain a first target head, tracking the first target head, and obtaining a first head image of the first target head. In the case that the first head image quality is qualified, face recognition and feature extraction are performed on the first head to obtain a facial feature value of the first target head. The face recognition and feature extraction are performed when the head image quality is qualified by judging the quality of the head image, preventing the situation of face recognition error caused by too small or not clear enough picture. The behavior state of the first target head is determined based on the tracking route of the first target head, the behavior state information of the first target head is determined based on the behavior state of the first target head, the facial feature value of the first target head and the behavior state information are saved to the passenger flow information statistical table, and the passenger flow information of the place to be counted is determined based on the passenger flow information statistical table. The accuracy of the passenger flow statistical result is improved.

[0016] In an optional implementation, the determining of the behavior state of the first target head based on the tracking route of the first target head comprises:

[0017] when the first target head disappears in the monitoring picture of the first camera, and the tracking route of the first target head in the monitoring picture is from top to bottom, the behavior state of the first target head is determined as an in-store state;

[0018] when the first target head disappears in the monitoring picture of the first camera, and the tracking route of the first target head in the monitoring picture is from bottom to top, the behavior state of the first target head is determined as an out-of-store state.

[0019] The passenger flow statistical method provided in the embodiment realizes accurate judgment of the in-store and out-of-store states of customers by determining the behavior state of the first target head based on the tracking route of the first target head, improves the accuracy and reliability of passenger flow statistics, provides more valuable customer behavior data for merchants, and helps to optimize operation strategies and improve sales performance.

[0020] In an optional implementation, the face recognition and feature extraction on the first head portrait are performed to obtain the face feature value of the first target head when the first head portrait is qualified in quality.

[0021] The quality of the first head portrait is judged by the following quality judgment condition:

[0022]

[0023]

[0024]

[0025]

[0026] wherein, W is a first head portrait width value, H is a first head portrait height value, Y is a number of pixels with a luminance value greater than 100 in the gray first head portrait, Gr(x, y) is a luminance value at a coordinate (x, y) in the gray first head portrait, Z is a number of pixels with a gradient value of an edge detection operator greater than 50 in the gray first head portrait, and G(x, y) is a gradient value of the edge detection operator at the coordinate (x, y) in the gray first head portrait.

[0027] When the first head portrait meets the quality judgment condition, it is determined that the first head portrait is qualified in quality, and the face recognition and feature extraction are performed on the first head portrait to obtain the face feature value of the first target head.

[0028] The traffic flow counting method provided in the embodiment ensures that only the first head portrait qualified in quality is subjected to face recognition and feature extraction through strict quality judgment conditions, which reduces the influence of low-quality images on the recognition result, thereby improving the accuracy and reliability of face recognition.

[0029] In an optional implementation, the method further comprises:

[0030] The gradient value of the horizontal direction edge detection operator and the gradient value of the vertical direction edge detection operator of the image block with the center coordinate (x, y) in the gray first head portrait are obtained by performing convolution operation on the gray first head portrait according to the following formula:

[0031]

[0032]

[0033] wherein, the gradient value of the horizontal direction edge detection operator of the image block with the center coordinate (x, y) in the gray first head portrait, a gradient value of a vertical edge detection operator of an image block with a center coordinate (x, y) in the gray first person head image, an image block with a center coordinate (x, y) in the gray first person head image;

[0034] A gradient value of an edge detection operator at coordinate (x, y) in the gray first person head image is determined based on the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator of the image block with the center coordinate (x, y) in the gray first person head image by the following formula:

[0035]

[0036] wherein G(x, y) is the gradient value of the edge detection operator at coordinate (x, y) in the gray first person head image.

[0037] The method for counting the number of visitors provided by the embodiment obtains the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator through convolution operation, and determines the final gradient value of the edge detection operator based on these gradient values, and then determines whether the first person head image meets the quality determination condition according to the gradient value of the edge detection operator, thereby ensuring the accuracy of the quality detection of the first person head image.

[0038] In an alternative embodiment, before saving the facial feature value of the first target person head and the behavior state information to the visitor flow information statistical table, the method further comprises:

[0039] collecting a face picture of an employee of the place to be counted;

[0040] extracting a feature value from the face picture of the employee to obtain an employee face feature value;

[0041] saving the employee face feature value to an employee face feature data table;

[0042] The saving of the facial feature value of the first target person head and the behavior state information to the visitor flow information statistical table comprises:

[0043] comparing the facial feature value of the first target person head with the employee face feature value in the employee face feature data table;

[0044] if the comparison fails, repeatedly comparing the facial feature value of the first target person head with the facial feature value in the visitor flow information statistical table;

[0045] If the repetitive comparison passes, the face feature value of the first target head and the behavior state information are updated to the storage area corresponding to the first target head in the passenger flow information statistical table;

[0046] If the repetitive comparison fails, a storage area of the first target head is created in the passenger flow information statistical table, and the face feature value of the first target head and the behavior state information are saved to the storage area of the first target head.

[0047] The passenger flow statistical method provided by the embodiment can exclude the case that an employee is mistaken for a customer, ensure that the passenger flow information in the passenger flow information statistical table only contains real customer data, and improve the accuracy of the statistical passenger flow information.

[0048] The repetitive comparison of the face feature value of the first target head with the face feature value in the passenger flow information statistical table ensures that the information of the same customer is not repeatedly counted, and improves the accuracy of the passenger flow statistical result.

[0049] In an optional implementation, the method further includes:

[0050] Obtaining second video data of a second camera in other areas of the place to be counted except the entrance;

[0051] Performing head detection on the second video data to obtain a second target head;

[0052] Tracking the second target head to obtain a second head image of the second target head;

[0053] Determining a tracking start time and a tracking end time based on the tracking route of the second target head;

[0054] Determining a stay time of the second target head under the second camera based on the tracking start time and the tracking end time;

[0055] In the case that the second head image is qualified, performing face recognition and feature extraction on the second head image to obtain a face feature value of the second target head;

[0056] Comparing the face feature value of the second target head with the employee face feature value in the employee face feature data table;

[0057] If the comparison fails, comparing the face feature value of the second target head with the face feature value in the passenger flow information statistical table repeatedly;

[0058] If the repetitive comparison passes, the face feature value of the second target head and the staying time of the second target head under the second camera are updated to the storage area corresponding to the second target head in the passenger flow information statistical table.

[0059] The passenger flow statistical method provided by the embodiment realizes statistical analysis on the behavior and activities of each customer after entering the store by acquiring the staying time of each customer under the second camera and associating the data in the plurality of second cameras through face feature value comparison, and provides important time dimension data for passenger flow analysis, helps the merchant to understand the staying situation of customers in different areas, and thus can analyze the interest degree of customers to different goods, facilitates the merchant to understand customers better, and thus optimizes the goods display, goods promotion and the like according to the preferences of customers.

[0060] In an optional implementation, the method further includes:

[0061] For any target head in the passenger flow information statistical table of the day, if the number of appearances of the target head in a historical preset time period reaches an appearance number threshold, the face feature value and the behavior state information of the target head are pushed to the employee terminal, so that the employee provides targeted service for the corresponding customer based on the face feature value and the behavior state information of the target head.

[0062] The passenger flow statistical method provided by the embodiment makes the employee pay more attention to the customers who repeatedly enter the store in a certain time by pushing the face feature value and the behavior state information of the target head that frequently appears to the employee, better serves these customers, and thus improves the purchase rate of goods.

[0063] In a second aspect, the present application provides a passenger flow statistical device, the device includes:

[0064] The first acquisition module is configured to acquire first video data of a first camera at the entrance of a place to be counted.

[0065] The second acquisition module is configured to perform head detection on the first video data to obtain a first target head.

[0066] The third acquisition module is configured to track the first target head to obtain a first head image of the first target head.

[0067] The first determination module is configured to determine the behavior state of the first target head based on the tracking route of the first target head.

[0068] The fourth acquisition module is configured to perform face recognition and feature extraction on the first head image to obtain the face feature value of the first target head if the first head image is qualified.

[0069] The second determining module is configured to determine behavior state information of the first target head based on the behavior state of the first target head.

[0070] The first saving module is configured to save the face feature value of the first target head and the behavior state information to a flow information statistical table.

[0071] The third determining module is configured to determine flow information of the place to be counted based on the flow information statistical table.

[0072] In a third aspect, the present application provides a computer device, comprising a memory and a processor, the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the flow counting method of the first aspect or any of the corresponding embodiments thereof.

[0073] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the flow counting method of the first aspect or any of the corresponding embodiments thereof.

[0074] In a fifth aspect, the present application provides a computer program product, which comprises computer instructions, and the computer instructions are used to make a computer execute the flow counting method of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the specific embodiments or the related art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the related art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0076] Figure 1 FIG. 1 is a flowchart of a flow counting method according to an embodiment of the present application;

[0077] Figure 2 FIG. 2 is a flowchart of another flow counting method according to an embodiment of the present application;

[0078] Figure 3 FIG. 3 is a flowchart of still another flow counting method according to an embodiment of the present application;

[0079] Figure 4 FIG. 4 is a structural block diagram of a flow counting device according to an embodiment of the present application;

[0080] Figure 5Fig. 1 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0081] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0082] Traditional retail and service industries rely on customer flow to measure sales performance. Through the statistics and analysis of customer flow, merchants can deeply understand the behavior patterns of customers in the store, such as customer flow, time spent in front of different goods, in-store flow routes, etc., so as to optimize staff allocation, product display, promotion planning, etc., improve operational efficiency and service quality, and ultimately achieve higher sales performance.

[0083] With the development and progress of artificial intelligence and video image analysis technologies, the accuracy and efficiency of customer flow statistics have been significantly improved. These technologies enable customer flow statistics to remain highly accurate in complex environments, providing more reliable data support for business decision-making.

[0084] In related technologies, there are various methods for customer flow statistics, such as infrared-based customer flow statistics, WIFI-based customer flow statistics, video-based customer flow statistics, etc., and the commonly used method is video-based customer flow statistics. There are various implementation methods for video-based customer flow statistics, such as face detection and tracking-based customer flow statistics, head detection and tracking-based customer flow statistics, etc.

[0085] However, the face detection and tracking-based customer flow statistics method does not consider the situation that the face cannot be recognized and tracked due to the turning of the head or the lowering of the head during tracking, resulting in low accuracy of the obtained customer flow statistics result. Moreover, the face detection and tracking-based customer flow statistics method lacks quality judgment of face images, and there are situations of face recognition errors caused by reasons such as too small or not clear enough face pictures.

[0086] The customer flow counting method based on head detection and tracking can only count the total number of people entering and leaving the place, and cannot count and analyze specific behavior data of customers such as entering time, leaving time, and staying time in front of different goods, and cannot count and analyze customers entering the store in a day and customers repeatedly entering the store in a month, and the like. These data are important factors affecting the sales performance of the store, and the customer flow analysis result is not accurate enough without counting these data, which affects the in-depth understanding of customer behavior by the store.

[0087] Moreover, the above two customer flow counting methods do not consider the influence of employees on the counting result, that is, the employee face feature value is not counted, updated, and compared, resulting in low accuracy of the obtained customer flow counting result.

[0088] The embodiment of the present application provides a customer flow counting method, which detects a first target head by detecting a head in first video data of a first camera at a door of a place to be counted, tracks the first target head, obtains a first head image of the first target head, performs face recognition and feature extraction on the first target head in the case that the first head image is qualified, obtains a face feature value of the first target head, determines a behavior state of the first target head based on a tracking route of the first target head, determines behavior state information of the first target head based on the behavior state of the first target head, saves the face feature value and the behavior state information of the first target head to a customer flow information counting table, and determines customer flow information of the place to be counted based on the customer flow information counting table, so as to prevent the face from being unable to be recognized and tracking from failing due to turning the head or lowering the head, prevent face recognition errors caused by too small or not clear enough pictures, and improve the accuracy of the customer flow counting result.

[0089] According to the embodiment of the present application, a customer flow counting method is provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0090] In this embodiment, a customer flow counting method is provided, which can be used in mobile terminals such as servers, central processing units, and the like, Figure 1 The flowchart of the customer flow counting method according to the embodiment of the present application is shown in FIG. Figure 1 The flowchart includes the following steps:

[0091] In step S101, first video data of a first camera at a door of a place to be counted is obtained.

[0092] The to-be-counted place can be a store of a retail industry and a service industry. First video data captured by a first camera for monitoring in-and-out store passenger flow at a store entrance of the to-be-counted place is acquired in real time.

[0093] In step S102, a first target head is obtained by performing head detection on the first video data.

[0094] In the method, after the first video data is acquired, a first target head in the first video data is obtained by performing head detection on the first video data based on a trained neural network model such as a yolov5 model.

[0095] In step S103, a first head image of the first target head is obtained by tracking the first target head.

[0096] In the method, after the first target head is detected, the first target head in the first video data is tracked, and a first head image of the first target head is obtained in the tracking process.

[0097] In step S104, a behavior state of the first target head is determined based on a tracking route of the first target head.

[0098] In the method, the first target head is tracked to obtain a tracking route of the first target head, and the behavior state of the first target head is determined based on the tracking route. The behavior state can be an in-store state or an out-store state.

[0099] In step S105, face recognition and feature extraction are performed on the first head image to obtain a face feature value of the first target head, in a case where the first head image is qualified.

[0100] In the method, after the first head image is obtained, quality judgment is performed on the first head image, and in a case where the first head image is qualified, face recognition and feature extraction, i.e., feature calculation, are performed on the first head image based on a deep learning method such as SeetaFace to obtain a face feature value of the first target head.

[0101] In step S106, behavior state information of the first target head is determined based on the behavior state of the first target head.

[0102] In a case where the behavior state of the first target head is an in-store state, the behavior state information of the first target head is determined to be an in-store time.

[0103] In a case where the behavior state of the first target head is an out-store state, the behavior state information of the first target head is determined to be an out-store time.

[0104] In step S107, the face feature value and the behavior state information of the first target head are saved to a passenger flow information statistical table.

[0105] wherein, after obtaining the facial feature value and the behavior state information of the first target head, the facial feature value and the behavior state information of the first target head are saved to the customer flow information statistical table.

[0106] It should be noted that the customer flow information statistical table of each day is established in the database according to the date, and is used to record the facial feature value, the time of entering the store, the time of leaving the store and the like of the customer of each day. These information can be used to determine the key customers who enter the store for multiple times in a period of time.

[0107] It can be understood that the facial feature value and the behavior state information of the first target head are saved to the corresponding customer flow information statistical table according to the behavior state information.

[0108] In step S108, the customer flow information of the place to be counted is determined based on the customer flow information statistical table.

[0109] Based on the customer flow information statistical table of each day, the customer flow information of the place to be counted of each day can be determined. The customer flow information, i.e. the customer flow statistical information, includes the number of customers entering and leaving the store, the time of entering and leaving the store of each customer and the like.

[0110] wherein, the number of customers entering and leaving the store of each day is determined according to the number of target heads corresponding to the facial feature value and the behavior state information in the customer flow information statistical table of each day.

[0111] The customer flow statistical method provided in the embodiment prevents the situation that the face cannot be recognized and the tracking fails due to the head turning or the head lowering by performing head detection on the first video data of the first camera at the entrance of the place to be counted to obtain the first target head, tracking the first target head to obtain the first head image of the first target head. In the case that the quality of the first head image is qualified, the face recognition and the feature extraction are performed on the first head to obtain the facial feature value of the first target head, and the face recognition and the feature extraction are performed when the quality of the head image is qualified by judging the quality of the head image, so as to prevent the situation that the face recognition is wrong due to the picture being too small or not clear enough. The behavior state of the first target head is determined based on the tracking route of the first target head, the behavior state information of the first target head is determined based on the behavior state of the first target head, the facial feature value and the behavior state information of the first target head are saved to the customer flow information statistical table, and the customer flow information of the place to be counted is determined based on the customer flow information statistical table. The accuracy of the customer flow statistical result and the analysis result of the customer flow statistical result is improved.

[0112] In the embodiment, a customer flow statistical method is provided, which can be used in a mobile terminal such as a server, a central processing unit and the like, Figure 2is a flowchart of a passenger flow counting method according to an embodiment of the present application, as shown in the figure, the flow includes the following steps: Figure 2

[0113] In step S201, first video data of a first camera at a gate of a place to be counted is acquired. For details, please refer to step S101 of the embodiment shown in the figure, which will not be repeated here. Figure 1

[0114] In step S202, head detection is performed on the first video data to obtain a first target head. For details, please refer to step S102 of the embodiment shown in the figure, which will not be repeated here. Figure 1

[0115] In step S203, tracking is performed on the first target head to obtain a first head image of the first target head. For details, please refer to step S103 of the embodiment shown in the figure, which will not be repeated here. Figure 1

[0116] In step S204, a behavior state of the first target head is determined based on a tracking route of the first target head.

[0117] Specifically, the above step S204 includes:

[0118] In step S2041, when the first target head disappears in the monitoring picture of the first camera, if the tracking route of the first target head in the monitoring picture is from top to bottom, it is determined that the behavior state of the first target head is a store-entering state.

[0119] It can be understood that the behavior state of the first target head is determined according to the tracking route of the first target head in the monitoring picture of the first camera.

[0120] The tracking route of the first target head in the monitoring picture is from top to bottom, which means that the first target head is from far to near until it disappears from the monitoring picture of the first camera, and it is determined that the behavior state of the first target head is a store-entering state, i.e., the customer corresponding to the first target head enters the store.

[0121] In step S2042, when the first target head disappears in the monitoring picture of the first camera, if the tracking route of the first target head in the monitoring picture is from bottom to top, it is determined that the behavior state of the first target head is a store-exiting state.

[0122] The tracking route of the first target head in the monitoring picture is from bottom to top, which means that the first target head is from near to far until it disappears from the monitoring picture of the first camera, and it is determined that the behavior state of the first target head is a store-exiting state, i.e., the customer corresponding to the first target head exits the store.

[0123] ​​​​Step S205, in the case that the first head image is qualified, performing face recognition and feature extraction on the first head image to obtain a face feature value of the first target head. For details, please refer to Figure 1 Step S105 of the embodiment shown will not be repeated here.

[0124] Step S206, determining behavior state information of the first target head based on the behavior state of the first target head. For details, please refer to Figure 1 Step S106 of the embodiment shown will not be repeated here.

[0125] Step S207, saving the face feature value and the behavior state information of the first target head to the flow information statistics table. For details, please refer to Figure 1 Step S107 of the embodiment shown will not be repeated here.

[0126] Step S208, determining flow information of the place to be counted based on the flow information statistics table. For details, please refer to Figure 1 Step S108 of the embodiment shown will not be repeated here.

[0127] The flow counting method provided in the embodiment determines the behavior state of the first target head based on the tracking route of the first target head, accurately judges the in-store and out-of-store state of the customer, improves the accuracy and reliability of flow counting, provides more valuable customer behavior data for the merchant, helps to optimize the operation strategy and improve the sales performance.

[0128] In some optional embodiments, the above step S205 includes:

[0129] Step a1, performing quality judgment on the first head image by the following quality judgment condition:

[0130]

[0131]

[0132]

[0133]

[0134] Wherein, W is a first head image width value, H is a first head image height value, Y is a number of pixels with a luminance value greater than 100 in the gray first head image, Gr(x, y) is a luminance value at coordinates (x, y) in the gray first head image, Z is a number of pixels with a gradient value of an edge detection operator greater than 50 in the gray first head image, and G(x, y) is a gradient value of the edge detection operator at coordinates (x, y) in the gray first head image.

[0135] It can be understood that the gray first-person head image is an image obtained by performing gray processing on the first-person head image.

[0136] In a case where the first-person head image meets the quality judgment condition, the first-person head image quality is determined to be qualified, and face recognition and feature extraction are performed on the first-person head image to obtain the face feature value of the first target person head.

[0137] The first-person head image meets the quality judgment condition, that is, the width of the first-person head image is greater than 80 pixels, the height of the first-person head image is greater than 80 pixels, and the sum of the number of pixels with a luminance value greater than 100 and the number of pixels with a gradient value (representing texture information) greater than 50 in the first-person head image accounts for more than half of the picture, and the first-person head image quality is determined to be qualified.

[0138] The passenger flow statistical method provided in the embodiment ensures that only the first-person head image with qualified quality is subjected to face recognition and feature extraction through strict quality judgment conditions, which reduces the influence of low-quality images on the recognition result, prevents inaccurate calculation of the face feature value due to excessively small pictures or insufficient clarity, and thus improves the accuracy and reliability of face recognition.

[0139] In some optional embodiments, the passenger flow statistical method further includes:

[0140] Step b1: performing convolution operation on the gray first-person head image through the following formula to obtain the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image:

[0141]

[0142]

[0143] wherein, is the gradient value of the horizontal edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image, is the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image, is the image block with the center coordinates (x, y) in the gray first-person head image.

[0144] Step b2: determining the gradient value of the edge detection operator at the coordinates (x, y) in the gray first-person head image based on the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image through the following formula:

[0145]

[0146] G(x, y) = |Gx(x, y)| + |Gy(x, y)|, wherein G(x, y) is a gradient value of the edge detection operator at the coordinate (x, y) in the gray first head image.

[0147] The traffic flow statistical method provided by the embodiment obtains the gradient value of the horizontal direction edge detection operator and the gradient value of the vertical direction edge detection operator through convolution operation, and determines the final gradient value of the edge detection operator based on the gradient values, and then determines whether the first head image meets the quality judgment condition according to the gradient value of the edge detection operator, thereby ensuring the accuracy of the quality detection of the first head image.

[0148] In some optional embodiments, before saving the face feature value of the first target head and the behavior state information to the traffic flow information statistical table, the traffic flow statistical method further comprises:

[0149] Step c1, collecting a face picture of an employee in the place to be counted.

[0150] Step c2, extracting a feature value from the face picture of the employee to obtain an employee face feature value.

[0151] The face feature value in the face picture of the employee is extracted by using a deep learning method such as SeetaFace to obtain the employee face feature value.

[0152] Step c3, saving the employee face feature value to an employee face feature data table.

[0153] After obtaining the employee face feature value, the employee face feature value is saved to an employee face feature value data table. The employee face feature data table is pre-established in the database to remove the influence of the employee on the passenger flow statistics and analysis.

[0154] It should be noted that one employee can include multiple face feature values, such as face feature values of the front face of the employee and face feature values of the side face of the employee, and the like. It can be understood that the employee face feature value data table actually stores the correspondence between the employee and the employee face feature value.

[0155] It can be understood that the employee face feature value data table can be supplemented and updated in a timely manner according to actual conditions.

[0156] The above step S207 comprises:

[0157] Step d1, comparing the face feature value of the first target head with the employee face feature value in the employee face feature data table.

[0158] Wherein, after obtaining the face feature value of the first target head, the face feature value of the first target head is compared with the face feature value of the employee in the employee face feature data table to determine whether the person corresponding to the first target head is an employee.

[0159] Step d2, if the comparison fails, the face feature value of the first target head is repeatedly compared with the face feature value in the customer flow information statistics table.

[0160] Wherein, if the face feature value of the first target head fails to compare with any face feature value of the employee in the employee face feature data table, it means that the person corresponding to the first target head is not an employee, and the face feature value of the first target head is repeatedly compared with the plurality of face feature values in the customer flow information statistics table of the day to determine whether the customer corresponding to the first target head has entered the store.

[0161] It can be understood that if the face feature value of the employee in the employee face feature data table is the same as the face feature value of the first target head, it is determined that the comparison is passed, which means that the person corresponding to the first target head is an employee, and the steps after the step do not need to be executed, and the process is ended.

[0162] Step d3, if the repeated comparison is passed, the face feature value and the behavior state information of the first target head are updated to the storage area corresponding to the first target head in the customer flow information statistics table.

[0163] Wherein, if the face feature value of the first target head is the same as the face feature value in the customer flow information statistics table of the day, it is determined that the repeated comparison is passed, the customer corresponding to the first target head has entered the store, and the face feature value and the behavior state information of the first target head are updated to the storage area corresponding to the first target head in the customer flow information statistics table of the day.

[0164] It should be noted that updating the face feature value and the behavior state information of the first target head to the storage area corresponding to the first target head in the customer flow information statistics table of the day includes:

[0165] If the face feature value of the first target head is the same as any face feature value in the storage area corresponding to the first target head, the face feature value of the first target head does not need to be updated to the storage area corresponding to the first target head, and only the behavior state information of the first target head needs to be added to the storage area corresponding to the first target head.

[0166] Step d4, if the repeated comparison fails, a storage area of the first target head is created in the customer flow information statistics table, and the face feature value and the behavior state information of the first target head are saved to the storage area of the first target head.

[0167] If the face feature value of the first target head is different from the face feature value of any one in the customer flow information statistical table, it is determined that the repetitive comparison fails, the first target head has not entered the store today, and a storage area of the first target head is created in the customer flow information statistical table, and the face feature value and the behavior state information of the first target head are saved to the storage area of the first target head.

[0168] It can be understood that the customer flow information statistical table substantially includes the correspondence between the first target head and the storage area corresponding to the first target head, and the storage area stores the face feature value and the behavior state information of the first target head.

[0169] That is, in the case that the behavior state of the first target head is the entering state, the face feature value of the first target head is compared with the employee face feature value in the employee face feature data table; if the comparison fails, the face feature value of the first target head is compared with the face feature value in the customer flow information statistical table; if the repetitive comparison passes, the face feature value and the entering time of the first target head are updated to the storage area corresponding to the first target head in the customer flow information statistical table; if the repetitive comparison fails, the storage area of the first target head is created in the customer flow information statistical table, and the face feature value and the entering time of the first target head are saved to the storage area of the first target head.

[0170] In the case that the behavior state of the first target head is the entering state, the face feature value of the first target head is compared with the employee face feature value in the employee face feature data table; if the comparison fails, the face feature value of the first target head is compared with the face feature value in the customer flow information statistical table; if the repetitive comparison passes, the face feature value and the entering time of the first target head are updated to the storage area corresponding to the first target head in the customer flow information statistical table; if the repetitive comparison fails, the storage area of the first target head is created in the customer flow information statistical table, and the face feature value and the entering time of the first target head are saved to the storage area of the first target head.

[0171] The customer flow statistical method provided by the embodiment can exclude the case that an employee is mistaken for a customer, ensure that the customer flow information in the customer flow information statistical table only contains real customer data, and improve the accuracy of the statistical customer flow information.

[0172] The face feature value of the first target head is compared with the face feature value in the customer flow information statistical table, so that the information of the same customer is not repeatedly counted, and the accuracy of the customer flow statistical result is improved.

[0173] In some optional embodiments, the passenger flow counting method further comprises:

[0174] Step e1, acquiring second video data of a second camera in other areas of the place to be counted except the entrance.

[0175] Wherein, the second video data shot by the second camera in other areas of the place to be counted except the entrance is acquired in real time.

[0176] Step e2, detecting the second video data for human head to obtain a second target human head.

[0177] Wherein, after acquiring the second video data, the second video data is detected for human head based on a trained neural network model such as a yolov5 model to obtain a second target human head in the second video data.

[0178] Step e3, tracking the second target human head to obtain a second human head image of the second target human head.

[0179] Wherein, after detecting the second target human head, the second target human head in the second video data is tracked, and in the tracking process, a second human head image of the second target human head is obtained.

[0180] Step e4, determining a tracking start time and a tracking end time based on the tracking route of the second target human head.

[0181] Tracking the second target human head to obtain a tracking route of the second target human head, and determining a tracking start time and a tracking end time of the second target human head under the second camera based on the tracking route.

[0182] When starting to track the second target human head, the tracking start time is recorded, and when the second target human head disappears in the monitoring screen of the second camera, the tracking end time is recorded.

[0183] Step e5, determining a stay time of the second target human head under the second camera based on the tracking start time and the tracking end time.

[0184] The stay time of the second target human head under the second camera is obtained by subtracting the tracking start time from the tracking end time.

[0185] Step e6, in the case that the quality of the second human head image is qualified, performing face recognition and feature extraction on the second human head image to obtain a face feature value of the second target human head.

[0186] Wherein, after obtaining the second human head image, the quality of the second human head image is judged, and the quality judgment method is similar to the quality judgment method of the first human head image, which is not described here.

[0187] In the case that the second head image quality is qualified, the second head image is subjected to face recognition and feature extraction based on a deep learning method such as SeetaFace, i.e. feature calculation, to obtain the face feature value of the second target head.

[0188] Step e7, the face feature value of the second target head is compared with the face feature value of the employee in the employee face feature data table. Similar to the description of step d1, it will not be repeated here.

[0189] Step e8, if the comparison fails, the face feature value of the second target head is repeatedly compared with the face feature value in the passenger flow information statistics table. Similar to the description of step d2, it will not be repeated here.

[0190] Step e9, if the repeated comparison is passed, the face feature value of the second target head and the stay time of the second target head under the second camera are updated to the storage area corresponding to the second target head in the passenger flow information statistics table.

[0191] If there is a face feature value in the passenger flow information statistics table that is the same as the face feature value of the second target head, it is determined that the repeated comparison is passed, the customer corresponding to the second target head has already entered the store, and the face feature value of the second target head and the stay time of the second target head under the second camera are updated to the storage area corresponding to the second target head in the passenger flow information statistics table.

[0192] It should be noted that updating the face feature value of the second target head and the stay time of the second target head under the second camera to the storage area corresponding to the second target head in the passenger flow information statistics table includes:

[0193] If the face feature value of the second target head is the same as any face feature value in the storage area corresponding to the second target head, the face feature value of the second target head does not need to be updated to the storage area corresponding to the second target head, but only the stay time of the second target head under the second camera needs to be added to the storage area corresponding to the second target head.

[0194] That is, the passenger flow information statistics table is also used to record the stay time of the customer under each camera, and the passenger flow information further includes the stay time of the customer under each camera, etc.

[0195] The customer flow statistical method provided in the embodiment realizes statistical analysis on the behavior activities of each customer after entering the store by counting the time of each customer after entering the store appearing in each camera, obtaining the staying time of each customer under the second camera, and associating the data in the plurality of second cameras through face feature value comparison, and further realizes statistical analysis on the behavior activities of all customers entering the store in a day, provides important time dimension data for customer flow analysis, helps the merchant to understand the staying situation of customers in different areas, and further can analyze the interest degree of customers to different goods, facilitates the merchant to understand the shopping preferences of customers more, and thus optimizes the goods display, staff allocation, goods promotion and the like according to the preferences of customers.

[0196] In some optional embodiments, the customer flow statistical method further comprises:

[0197] In step f1, for any target head in the customer flow information statistical table of the day, if the number of appearances of the target head in the historical preset time period reaches the appearance number threshold, the face feature value and the behavior state information of the target head are pushed to the staff terminal, so that the staff provides targeted service for the corresponding customer based on the face feature value and the behavior state information of the target head.

[0198] In the embodiment, the historical preset time period can be one month in the past. The appearance number threshold is determined by a technician. For example, the appearance number threshold is 2 times.

[0199] For any target head in the customer flow information statistical table of the day, the customer flow information statistical table of the past one month is queried, and if the number of appearances of the target head in the customer flow information statistical table of the past one month reaches the appearance number threshold, the face feature value and the behavior state information stored in the storage area of the target head in the customer flow information statistical table are pushed to the staff terminal through the mini-program pushing mode, so that the staff provides service for the customer based on these information.

[0200] It should be noted that the staying time of the target head under the second camera stored in the storage area can also be synchronized to the staff.

[0201] The customer flow statistical method provided in the embodiment queries the historical entering store records of customers of the day, and when the key customers who enter the store multiple times in a certain time are queried, the face feature value and the behavior state information of the frequently appearing target head are pushed to the staff through the mini-program pushing mode, reminding the staff in the store to pay attention in time, so that the staff pays attention to the customers who enter the store repeatedly in a certain time, better serves these customers, and further improves the purchase rate of goods.

[0202] In some optional embodiments, the customer flow statistical method further comprises:

[0203] Step g1, for any target head in the passenger flow information statistics table of the day, obtain the stay time of the target head under the second camera.

[0204] Step g2, if the stay time of the target head under the second camera exceeds the stay time threshold, push the face feature value, behavior state information and stay time of the target head under the second camera to the employee terminal of the employee managing the monitoring area of the second camera, so that the employee managing the monitoring area of the second camera provides targeted service for the corresponding customer based on the face feature value, behavior state information and stay time of the target head under the second camera.

[0205] The stay time threshold is set by a technician and is not limited here.

[0206] If the stay time of any target head under the second camera exceeds the stay time threshold, it means that there is a product of interest to the target head in the monitoring area of the second camera, and the face feature value, behavior state information and stay time of the target head under the second camera are pushed to the employee terminal of the employee managing the monitoring area of the second camera, so that the employee provides targeted service to the customer.

[0207] The passenger flow statistics method provided in the embodiment compares the stay time of the target head under the second camera with the stay time threshold, and if the stay time of the target head under the second camera exceeds the stay time threshold, the employee managing the monitoring area of the second camera is reminded through the message pushing of the applet, so that the employee can more quickly identify customers who may need help, thereby providing more timely and efficient service and improving the shopping experience of customers.

[0208] In the embodiment, a passenger flow statistics method is provided, which can be used in mobile terminals such as servers, central processing units, etc. Figure 3 The flowchart of the passenger flow statistics method according to the embodiment of the application is shown in FIG. 1, which includes the following steps: Figure 3

[0209] First, collect the employee face information of the store and establish a feature value database.

[0210] The employee face information of the store is collected, and the face feature value is extracted to establish an employee face feature database. The feature value database, i.e., the employee face feature database, corresponds to the employee face feature data table in the foregoing embodiment.

[0211] The detailed description of this step is described in the foregoing embodiment steps c1 to c3, which will not be repeated here.

[0212] Second, establish a passenger flow information database for the day.​

[0213] The customer flow information database corresponds to the customer flow information statistics table in the foregoing embodiments. The customer flow information database is used to record the facial feature value, the time of entering the store, the time of leaving the store, the time of staying under each camera, and the like of the customer of the day.

[0214] In the third step, the independent thread 1 is started, the camera monitoring the entrance and exit of the store is used to detect and track the head, the head image quality is judged to determine whether the face recognition can be performed, the face recognition and the feature value calculation are performed in the case that the face recognition can be performed, it is determined whether a person enters the store, the face feature value recognized is compared with the employee face feature value in the case that a person enters the store, it is determined whether the person is an employee, the face feature of the customer of the day is compared in the case that the person is not an employee, it is determined whether there is repetition, the entering time is increased in the case that there is repetition, and the new customer is added in the case that there is no repetition.

[0215] Specifically, the camera monitoring the customer flow of the store entrance and exit is used to detect and track the head, the head image is judged according to the quality judgment condition to determine the quality in the tracking process, the face recognition and the feature calculation are performed on the picture meeting the quality to obtain the face feature value, it is determined whether the customer enters or exits the store according to the tracking route, the face feature value of the customer entering the store is compared, the employee face feature value is compared to determine whether the person is an employee, the customer of the day in the database is compared to determine whether there is repetition in the case that the person is not an employee, the new customer entering the store is added, the entering time and the face feature value are saved in the database.

[0216] It can be understood that only the process of counting the customer flow of the customer entering the store is shown in the figure, the process of counting the customer flow of the customer exiting the store is similar to the process of counting the customer flow of the customer entering the store, and details are not described herein.

[0217] For details, refer to the related description of the foregoing step S202 to the foregoing step S207, and details are not described herein.

[0218] In the fourth step, the independent thread 2 is started at the same time that the independent thread 1 is started, the other cameras in the store are used to detect and track the head, the head image quality is judged to determine whether the face recognition can be performed, the face recognition and the feature value calculation are performed in the case that the face recognition can be performed, it is determined whether a person exits the camera, the face feature value recognized is compared with the employee face feature to determine whether the person is an employee in the case that a person exits the camera, the face feature of the customer of the day is compared to determine whether there is an entering record in the case that the person is not an employee, and the staying time of the customer under the camera is recorded in the case that there is an entering record.

[0219] It can be understood that this step has the same meaning as steps e1 to e9 of the preceding embodiments, and details are referred to the detailed description of steps e1 to e9 of the preceding embodiments, which will not be repeated here.

[0220] In the fifth step, independent thread 3 is started while independent thread 1 and independent thread 2 are started, and the customer who enters the store on the day is queried for the historical record of entering the store, and it is judged whether there is a historical record of entering the store; if yes, a message is pushed through the applet to remind the staff.

[0221] It can be understood that this step has the same meaning as step f1 of the preceding embodiments, and details are referred to the detailed description of step f1 of the preceding embodiments, which will not be repeated here.

[0222] The customer flow statistical method provided in the embodiment is based on head detection and tracking, face picture quality judgment, face feature calculation and comparison, so that the customer flow statistical result is more accurate. The time of the customer entering the store under each camera is counted, the shopping preferences of the customer can be analyzed, and data support is provided for commodity display, staff configuration and the like of the merchant. The key customers who repeatedly enter the store within a certain time are paid attention to and reminded, so that these customers are better served, and the commodity purchase rate and the like are improved.

[0223] In the embodiment, a customer flow statistical device is also provided, which is used to implement the above-described embodiments and preferred embodiments, and details have been described and will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0224] The embodiment provides a customer flow statistical device, as shown in the following figure, which comprises: Figure 4

[0225] The first acquisition module 401 is configured to acquire first video data of a first camera at the entrance of a place to be counted.

[0226] The second acquisition module 402 is configured to perform head detection on the first video data to obtain a first target head.

[0227] The third acquisition module 403 is configured to track the first target head to obtain a first head image of the first target head.

[0228] The first determination module 404 is configured to determine the behavior state of the first target head based on the tracking route of the first target head.

[0229] The fourth acquisition module 405 is configured to perform face recognition and feature extraction on the first head image to obtain a face feature value of the first target head if the first head image is qualified.​

[0230] The second determining module 406 is configured to determine behavior state information of the first target head based on the behavior state of the first target head.

[0231] The first saving module 407 is configured to save the face feature value and the behavior state information of the first target head to the flow information statistics table.

[0232] The third determining module 408 is configured to determine flow information of the place to be counted based on the flow information statistics table.

[0233] In some optional embodiments, the first determining module 404 includes:

[0234] The first determining unit is configured to determine that the behavior state of the first target head is an in-store state when the first target head disappears in the monitoring picture of the first camera and the tracking route of the first target head in the monitoring picture is from top to bottom.

[0235] The second determining unit is configured to determine that the behavior state of the first target head is an out-of-store state when the first target head disappears in the monitoring picture of the first camera and the tracking route of the first target head in the monitoring picture is from bottom to top.

[0236] In some optional embodiments, the fourth obtaining module 405 includes:

[0237] The first judging unit is configured to judge the quality of the first head image according to the following quality judgment condition:

[0238]

[0239]

[0240]

[0241]

[0242] Wherein, W is a first head image width value, H is a first head image height value, Y is a number of pixels with a luminance value greater than 100 in the gray first head image, Gr(x, y) is a luminance value at coordinates (x, y) in the gray first head image, Z is a number of pixels with a gradient value of an edge detection operator greater than 50 in the gray first head image, and G(x, y) is a gradient value of the edge detection operator at coordinates (x, y) in the gray first head image.

[0243] The first obtaining unit is configured to determine that the quality of the first head image is qualified when the first head image meets the quality judgment condition, perform face recognition and feature extraction on the first head image, and obtain the face feature value of the first target head.

[0244] In some optional embodiments, the passenger flow counting device further comprises:

[0245] The fifth obtaining module is configured to perform a convolution operation on the gray first-person head image according to the following formula to obtain the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image:

[0246]

[0247]

[0248] wherein, is the gradient value of the horizontal edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image, is the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image, is the image block with the center coordinates (x, y) in the gray first-person head image.

[0249] The fourth determining module is configured to determine the gradient value of the edge detection operator at the coordinates (x, y) in the gray first-person head image based on the gradient value of the horizontal edge detection operator and the gradient value of the vertical edge detection operator of the image block with the center coordinates (x, y) in the gray first-person head image according to the following formula:

[0250]

[0251] wherein, G(x, y) is the gradient value of the edge detection operator at the coordinates (x, y) in the gray first-person head image.

[0252] In some optional embodiments, the passenger flow counting device further comprises:

[0253] The collecting module is configured to collect the face pictures of the employees in the place to be counted.

[0254] The sixth obtaining module is configured to perform feature value extraction on the face pictures of the employees to obtain employee face feature values.

[0255] The second saving module is configured to save the employee face feature values to an employee face feature data table.

[0256] The first saving module 407 comprises:

[0257] The first comparing unit is configured to compare the face feature value of the first target head with the employee face feature values in the employee face feature data table.

[0258] The second comparison unit is configured to, if the comparison fails, repeatedly compare the face feature value of the first target head with the face feature value in the passenger flow information statistical table.

[0259] The first updating unit is configured to, if the repeated comparison passes, update the face feature value of the first target head and the behavior state information to the storage area corresponding to the first target head in the passenger flow information statistical table.

[0260] The first saving sub-unit is configured to, if the repeated comparison fails, create a storage area of the first target head in the passenger flow information statistical table, and save the face feature value of the first target head and the behavior state information to the storage area of the first target head.

[0261] In some optional embodiments, the passenger flow statistical device further comprises:

[0262] The seventh acquisition module is configured to acquire second video data of a second camera in an area other than the entrance of the place to be counted.

[0263] The eighth acquisition module is configured to perform head detection on the second video data to obtain a second target head.

[0264] The ninth acquisition module is configured to track the second target head to obtain a second head image of the second target head.

[0265] The fifth determination module is configured to determine a tracking start time and a tracking end time based on a tracking route of the second target head.

[0266] The sixth determination module is configured to determine a stay time of the second target head under the second camera based on the tracking start time and the tracking end time.

[0267] The tenth acquisition module is configured to, in the case that the second head image is qualified, perform face recognition and feature extraction on the second head image to obtain a face feature value of the second target head.

[0268] The first comparison module is configured to compare the face feature value of the second target head with an employee face feature value in an employee face feature data table.

[0269] The second comparison module is configured to, if the comparison fails, repeatedly compare the face feature value of the second target head with the face feature value in the passenger flow information statistical table.

[0270] The first updating module is configured to, if the repeated comparison passes, update the face feature value of the second target head and the stay time of the second target head under the second camera to the storage area corresponding to the second target head in the passenger flow information statistical table.

[0271] In some optional embodiments, the passenger flow statistics device further comprises:

[0272] The pushing module is configured to, for any target head in the passenger flow information statistics table of the day, if the number of occurrences of the target head in the historical preset time period reaches the number of occurrence threshold, push the face feature value and the behavior state information of the target head to the staff terminal, so that the staff provides targeted service for the corresponding customer based on the face feature value and the behavior state information of the target head.

[0273] Further function descriptions of the above-mentioned various modules and units are the same as those of the corresponding embodiments, and will not be described here.

[0274] The passenger flow statistics device in the embodiment is presented in the form of a functional unit. The unit herein refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0275] The embodiment of the application further provides a computer device having the passenger flow statistics device described above. Figure 4

[0276] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of a computer device provided by an optional embodiment of the application, as Figure 5 shown, the computer device includes one or more processors 501, a memory 502, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by using different buses, and can be installed on a common mainboard or in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple storage devices. Similarly, multiple computer devices can be connected, each device providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 In the computer device, the processor 501 is taken as an example.

[0277] ​The processor 501 can be a central processor, a network processor, or a combination thereof. The processor 501 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0278] The memory 502 stores instructions executable by the at least one processor 501 to cause the at least one processor 501 to perform the methods illustrated in the above embodiments.

[0279] The memory 502 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 502 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some optional embodiments, the memory 502 can optionally include a memory disposed remotely from the processor 501, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0280] The memory 502 can include a volatile memory such as a random access memory, and can also include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk. The memory 502 can further include a combination of the above-mentioned types of memories.

[0281] The computer device further includes a communication interface 503 for communication of the computer device with other devices or communication networks.

[0282] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0283] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, the operation of the computer can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc. Correspondingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0284] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for counting passenger flow, characterized in that, The method includes: Obtain the first video data from the first camera at the entrance of the location to be analyzed; Perform head detection on the first video data to obtain the first target head; Track the first target head to obtain a first head image of the first target head; Based on the tracking route of the first target head, determine the behavioral state of the first target head; If the quality of the first head image is acceptable, face recognition and feature extraction are performed on the first head image to obtain the face feature value of the first target head; Based on the behavioral state of the first target head, determine the behavioral state information of the first target head; Save the facial feature value of the first target head and the behavioral status information to the passenger flow information statistics table; Based on the aforementioned passenger flow information statistics table, determine the passenger flow information of the location to be statistically analyzed; The step of performing face recognition and feature extraction on the first head image, under the condition that the quality of the first head image is qualified, to obtain the face feature value of the first target head, includes: The quality of the first head image is assessed using the following quality assessment criteria: Where W is the width of the first head image, H is the height of the first head image, Y is the number of pixels with a brightness value greater than 100 in the grayscale first head image, Gr(x,y) is the brightness value at coordinates (x,y) in the grayscale first head image, Z is the number of pixels with a gradient value greater than 50 in the grayscale first head image, G(x,y) is the gradient value of the edge detection operator at coordinates (x,y) in the grayscale first head image, and the grayscale first head image is the image after grayscale processing of the first head image; If the first head image meets the quality judgment conditions, the quality of the first head image is determined to be qualified, and face recognition and feature extraction are performed on the first head image to obtain the face feature value of the first target head. The method further includes: The gradient values ​​of the horizontal and vertical edge detection operators of the image block with center coordinates (x, y) in the grayscale first head image are obtained by performing a convolution operation on the image using the following formulas: in, Let be the gradient value of the edge detection operator in the horizontal direction of the image patch with center coordinates (x, y) in the grayscale first head image. Let be the gradient value of the edge detection operator in the vertical direction of the image patch with center coordinates (x, y) in the grayscale first head image. The image block with center coordinates (x, y) in the grayscale first human head image; The gradient value of the edge detection operator at coordinates (x, y) in the grayscale first head image is determined using the following formula, based on the gradient values ​​of the horizontal and vertical edge detection operators of the image block with center coordinates (x, y) in the grayscale first head image: Where G(x, y) is the gradient value of the edge detection operator at coordinates (x, y) in the grayscale first head image.

2. The method according to claim 1, characterized in that, The step of determining the behavioral state of the first target head based on the tracking route of the first target head includes: When the first target head disappears from the monitoring screen of the first camera, and the tracking route of the first target head in the monitoring screen is from top to bottom, the behavior state of the first target head is determined to be the state of entering the store. When the first target head disappears from the monitoring screen of the first camera, and the tracking route of the first target head in the monitoring screen is from bottom to top, the behavior state of the first target head is determined to be the exiting state.

3. The method according to claim 1, characterized in that, Before saving the facial feature value of the first target head and the behavioral status information to the passenger flow information statistics table, the method further includes: Collect facial images of employees at the locations to be surveyed; Feature values ​​are extracted from the employee's facial image to obtain the employee's facial feature values; Save the employee's facial feature values ​​to the employee facial feature data table; The step of saving the facial feature value of the first target head and the behavioral status information to the passenger flow information statistics table includes: The facial feature value of the first target head is compared with the facial feature values ​​of the employees in the employee facial feature data table; If the comparison fails, the facial feature value of the first target head will be repeatedly compared with the facial feature value in the passenger flow information statistics table; If the repeatability comparison passes, the facial feature value of the first target head and the behavioral status information are updated to the storage area corresponding to the first target head in the passenger flow information statistics table; If the repeatability comparison fails, a storage area for the first target head is created in the passenger flow information statistics table, and the facial feature value of the first target head and the behavior status information are saved to the storage area of ​​the first target head.

4. The method according to claim 1, characterized in that, The method further includes: Acquire the second video data from the second camera in the area of ​​the location to be analyzed, excluding the entrance; Perform head detection on the second video data to obtain the second target head; Track the second target head to obtain a second head image of the second target head; Based on the tracking route of the second target head, determine the tracking start time and tracking end time; Based on the tracking start time and tracking end time, determine the duration of time the second target head remains under the second camera; If the quality of the second head image is acceptable, face recognition and feature extraction are performed on the second head image to obtain the face feature value of the second target head; The facial feature value of the second target head is compared with the facial feature value of the employees in the employee facial feature data table; If the comparison fails, the facial feature value of the second target head will be repeatedly compared with the facial feature value in the passenger flow information statistics table; If the repeatability comparison passes, the facial feature value of the second target head and the dwell time of the second target head under the second camera are updated to the storage area corresponding to the second target head in the passenger flow information statistics table.

5. The method according to claim 1, characterized in that, The method further includes: For any target person in the daily customer traffic statistics table, if the number of times the target person appears within a preset historical time period reaches a threshold, the facial feature value and behavioral status information of the target person will be pushed to the employee terminal so that the employee can provide targeted services to the corresponding customer based on the facial feature value and behavioral status information of the target person.

6. A passenger flow counting device, characterized in that, The device includes: The first acquisition module is used to acquire the first video data of the first camera at the entrance of the location to be statistically analyzed. The second acquisition module is used to perform head detection on the first video data to obtain the first target head; The third acquisition module is used to track the first target head and obtain a first head image of the first target head; The first determining module is used to determine the behavioral state of the first target head based on the tracking route of the first target head; The fourth acquisition module is used to perform face recognition and feature extraction on the first head image when the quality of the first head image is qualified, so as to obtain the face feature value of the first target head. The second determining module is used to determine the behavior state information of the first target head based on the behavior state of the first target head; The first storage module is used to save the facial feature value of the first target head and the behavioral status information to the passenger flow information statistics table; The third determining module is used to determine the passenger flow information of the location to be counted based on the passenger flow information statistics table; The fourth acquisition module includes: The first judgment unit is used to judge the quality of the first head image according to the following quality judgment conditions: Where W is the width of the first head image, H is the height of the first head image, Y is the number of pixels with a brightness value greater than 100 in the grayscale first head image, Gr(x,y) is the brightness value at coordinates (x,y) in the grayscale first head image, Z is the number of pixels with a gradient value greater than 50 in the grayscale first head image, G(x,y) is the gradient value of the edge detection operator at coordinates (x,y) in the grayscale first head image, and the grayscale first head image is the image after grayscale processing of the first head image; The first acquisition unit is used to determine that the quality of the first head image is qualified when the first head image meets the quality judgment conditions, and to perform face recognition and feature extraction on the first head image to obtain the face feature value of the first target head. The passenger flow counting device also includes: The fifth acquisition module is used to perform a convolution operation on the grayscale first head image using the following formula to obtain the gradient values ​​of the horizontal and vertical edge detection operators for the image block with center coordinates (x, y) in the grayscale first head image: in, Let be the gradient value of the edge detection operator in the horizontal direction of the image patch with center coordinates (x, y) in the grayscale first head image. Let be the gradient value of the edge detection operator in the vertical direction of the image patch with center coordinates (x, y) in the grayscale first-level head image. The image patch with center coordinates (x, y) in the grayscale first human head image; The fourth determination module is used to determine the gradient value of the edge detection operator at coordinates (x, y) in the grayscale first head image based on the gradient values ​​of the horizontal and vertical edge detection operators of the image block with center coordinates (x, y) in the grayscale first head image, using the following formula: Where G(x, y) is the gradient value of the edge detection operator at coordinates (x, y) in the grayscale first head image.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the passenger flow statistics method according to any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the passenger flow statistics method according to any one of claims 1 to 5.

Images