A passenger flow counting method and system based on face recognition

Abstract

The present invention provides a passenger flow statistics method and system based on face recognition, including obtaining a face image captured by an image acquisition device, extracting face features using a face recognition engine and constructing a main picture set of face images based on the face features; merging face images with newly added face features into the corresponding main picture set of gathered people, otherwise, adding face images with newly added face features into the main picture set of gathered people; adding or updating the attribute information of the set; associating the sales commodity records of the store, the main picture set of gathered people and its attribute information, and obtaining the passenger flow statistics of the store. The method uses face attributes and face clustering applications to perform passenger flow statistics on stores or shops, and automatically obtains more accurate analysis data information of stores or shops every month.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a passenger flow counting method and system based on face recognition. Background Art

[0002] In recent years, the popularity of online e-commerce is gradually changing people's consumption habits. The competition between online and offline is becoming increasingly fierce. Offline operation costs are higher and the coverage is smaller. In the statistics of store or shop traffic, most of them are currently just simple headcount calculations. The big data of passenger flow statistics is to achieve simple traffic counts. There is no clearer data analysis, and it is impossible to analyze the store or store passenger flow data information or activity effects more clearly. With the development of related technologies such as face recognition and face detection, face applications are becoming more and more abundant. The face recognition passenger flow statistics system will record the image of customers, accurately, quickly and efficiently combine big data with offline stores, and lead a new round of changes in offline stores through passenger flow big data, member management big data, product sales big data, and advertising big data. Therefore, offline stores or shops urgently need face recognition passenger flow statistics to provide reference assistance to optimize the operation of stores or shops. Summary of the invention

[0003] In order to solve the technical problem that passenger flow statistics in the prior art are simple person-time flow technologies and cannot clearly analyze shop or store passenger flow data information or activity effects, the present invention proposes a passenger flow statistics method and system based on face recognition.

[0004] In one aspect, the present invention proposes a passenger flow counting method based on face recognition, comprising the following steps:

[0005] S1: Obtain a facial image captured by an image acquisition device, extract facial features using a facial recognition engine, and construct a main image set of aggregated persons of facial images based on the facial features;

[0006] S2: In response to the similarity between the newly added facial features and the facial features in the main image set of the gathered personnel being greater than the first threshold, or the similarity being between the first threshold and the second threshold and the number of facial features of the randomly selected facial images from the main image set of the gathered personnel being greater than the third threshold being greater than half, the facial image of the newly added facial features is merged into the corresponding main image set of the gathered personnel; otherwise, the facial image of the newly added facial features is added to the main image set of the gathered personnel;

[0007] S3: adding or updating attribute information of the collection, wherein the attribute information includes age, gender, store entry status and appearance time; and

[0008] S4: Associate the sales product records of the store, the main image set of the gathered people and its attribute information to obtain the customer flow statistics of the store.

[0009] Preferably, the image acquisition device includes an in-store capture device and an out-of-store capture device, and the store entry status is determined based on: in response to the face image being acquired by the in-store capture device, the store entry status of the face image is entered, otherwise it is not entered. The store entry status can be quickly determined using the images acquired by the in-store capture device.

[0010] Preferably, step S1 further includes obtaining a quality score of the face image and filtering face images below a face image quality score threshold. Filtering low-quality face images can improve the accuracy of face feature aggregation.

[0011] Further preferably, the quality score of the face image is specifically expressed as a weighted superposition of the scores of the face image pixel value, face ratio value, black and white gray pixel ratio value, eye number value, image clarity and face angle. A more accurate face image quality score can be obtained by weighted superposition of the scores.

[0012] Preferably, the specific method of updating age in step S3 is:

[0013] In response to the number of face images in the subset of the main set of aggregated personnel images being less than a preset number, taking an average value as the age of the subset;

[0014] In response to the number of face images in the subset of the main set of aggregated personnel images being greater than a preset number, bad values ​​containing large error values ​​are filtered out and an average value is taken as the age of the subset.

[0015] Further preferably, the bad value including the gross error value is specifically: if the residual error of the measurement value Nx_i satisfies |Vi|=|Nx_i-agv|<3σ, then the value is a bad value, where agv represents the arithmetic mean of the age of the subset, and σ is the standard deviation calculated using the Bessel formula.

[0016] Preferably, the specific method of updating the gender in step S3 is: the gender with a higher proportion in the subset of the main graph set of the clustered personnel is the gender of the subset.

[0017] Preferably, the customer flow statistics include time-based basic statistics and set attribute statistics, wherein the basic statistics include the total number of captured face images, the number of main sets of aggregated people, the number of sub-sets of people entering the store, the store entry rate, and the purchase rate. The diversification of customer flow statistics makes the analyzed data more complete, which is convenient for stores or shops to analyze.

[0018] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, the above method is implemented.

[0019] According to a third aspect of the present invention, a passenger flow counting system based on face recognition is proposed, characterized in that the system comprises:

[0020] A main image collection construction unit for gathering people: configured to obtain a face image captured by an image acquisition device, extract face features using a face recognition engine, and construct a main image collection of gathering people of face images based on the face features;

[0021] A judgment unit: configured to respond to the situation that the similarity between the newly added facial features and the facial features in the main image set of the gathered personnel is greater than a first threshold, or the similarity is between the first threshold and the second threshold and the number of facial features of the randomly selected facial images from the main image set of the gathered personnel that are greater than a third threshold is greater than half, merging the facial image with the newly added facial features into the corresponding main image set of the gathered personnel; otherwise, adding the facial image with the newly added facial features into the main image set of the gathered personnel;

[0022] Attribute information maintenance unit: configured to add or update attribute information of a collection, wherein the attribute information includes age, gender, store entry status and appearance time; and

[0023] Statistics unit: It is used to configure the sales product records of associated stores, the main image collection of aggregated personnel and its attribute information, and obtain the customer flow statistics of the store.

[0024] The present invention proposes a passenger flow statistics method and system based on face recognition, which uses a face recognition engine to extract face features and builds a main picture set of people gathering face images based on face features, adds or updates the age, gender, store entry status and appearance time information of the set; associates the sales commodity records of the store, the main picture set of people gathering and its attribute information, and finally obtains the passenger flow statistics of the store and displays the yesterday, the past week, the past month, and the fixed passenger flow statistics. The method or system can solve the problem of displaying the statistical information of the passenger flow to the personnel dimension (including gender and age) of shops and stores without feeling, instead of simple headcount or manpower statistics, so as to provide reference assistance for shops or stores to understand the passenger flow situation or activity effect, and optimize the operation of shops or stores. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0026] Figure 1 is a flow chart of a passenger flow counting method based on face recognition according to an embodiment of the present application;

[0027] Figure 2 is a flowchart of a passenger flow analysis method based on face recognition in a specific embodiment of the present application;

[0028] Figure 3 This is a flowchart of passenger flow classification statistics based on face recognition in a specific embodiment of the present application;

[0029] Figure 4 This is a framework diagram of a passenger flow counting system based on face recognition according to an embodiment of the present application;

[0030] Figure 5 It is a functional structure diagram of a passenger flow counting system based on face recognition in a specific embodiment of the present application;

[0031] Figure 6 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] According to a passenger flow counting method based on face recognition in one embodiment of the present application, Figure 1 FIG. 1 shows a flow chart of a passenger flow counting method based on face recognition according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0035] S101: Obtain a facial image captured by an image acquisition device, extract facial features using a facial recognition engine, and construct a main image set of aggregated persons of facial images based on the facial features.

[0036] In a specific embodiment, the image acquisition device includes an in-store capture device and an out-store capture device, and the above two capture devices are used to judge the store entry status. The specific judgment method is: if the face image is acquired by the in-store capture device, the store entry status of the face image is the store entry status, otherwise it is the non-store entry status.

[0037] In a specific embodiment, after obtaining the face image captured by the image acquisition device, the quality score of the face image can also be judged. The face image quality score is specifically the weighted superposition value of the pixel value of the face image, the face ratio value, the black and white gray pixel ratio value, the number of eyes value, the image clarity and the face angle. And the face image with a face image score lower than the preset quality score threshold is filtered, and the invalid or low-quality face images are eliminated to reduce the system's computing power and improve the accuracy of face recognition.

[0038] S102: In response to the newly added facial features having a similarity greater than a first threshold with the facial features in the main set of gathered people, or the similarity being between the first threshold and the second threshold and the number of facial features of randomly selected facial images from the main set of gathered people having a similarity greater than a third threshold being greater than half, the facial image of the newly added facial features is merged into the corresponding main set of gathered people, otherwise, the facial image of the newly added facial features is added to the main set of gathered people. The facial features of the captured images are clustered using the facial feature similarity to facilitate data analysis of the facial features.

[0039] S103: Add or update the attribute information of the collection, wherein the attribute information includes age, gender, store entry status and appearance time. With this step, the attribute information in the main picture collection of the gathered people can be added or updated, which is convenient for statistical and big data analysis.

[0040] In a specific embodiment, the gender status can be identified and analyzed by computer vision. The main gender recognition methods currently include: gender recognition algorithm based on feature face, gender recognition method based on Fisher criterion and face gender classification algorithm based on Adaboost+SVM. Age recognition can be estimated based on the age of the face image, using the skin texture, skin color, brightness and wrinkle texture in the face image, using CNN structure and SVM classifier to output the corresponding result.

[0041] In a specific embodiment, the specific implementation method for updating or adding age is: if the number of face images in the subset of the main image set of the gathered personnel is less than the preset number, the average value is taken as the age of the subset; if the number of face images in the subset of the main image set of the gathered personnel is greater than the preset number, the bad values ​​containing gross error values ​​are filtered out and the average value is taken as the age of the subset. Among them, the bad value containing gross error values ​​is specifically: if the residual error of the measured value Nx_i satisfies |Vi|=|Nx_i-agv|<3σ, then the value is a bad value, where agv represents the arithmetic mean of the age of the subset, and σ is the standard deviation calculated using the Bessel formula.

[0042] In a specific embodiment, the specific method of updating gender is: the gender of the subset with the highest proportion in the main set of the gathered personnel image is the gender of the subset. The gender information of the main set of the gathered personnel image can be quickly obtained through the proportion to avoid misjudgment.

[0043] S104: Associate the sales records of the store, the main image set of the gathered people and its attribute information to obtain the customer flow statistics of the store. The customer flow statistics data specifically include basic statistics based on time and set attribute statistics, wherein the basic statistics include the total number of captured face images, the number of main image sets of gathered people, the number of sub-sets of people entering the store, the store entry rate and the purchase rate.

[0044] Figure 2 A flow chart of a passenger flow analysis method based on face recognition according to a specific embodiment of the present application is shown. Figure 2 As shown, the analysis method includes:

[0045] S201: Start.

[0046] S202: Obtain captured face image, capture time, device ID, device name, and device type. According to the GAT-1400 public security video image information application system standard, class A and class B data are aggregated, and the analysis server obtains captured face image, scene image, capture time, capture device ID, device name, device type (class A or class B), and other data.

[0047] S203: Calculate the quality of the captured face image. Perform a quality analysis on the captured face image, detect the face image based on the image pixel value, face ratio value, black and white pixel ratio, eye detection quantity value, image clarity and face angle, and calculate the comprehensive quality score through a mathematical model. Specifically, the comprehensive quality score can be obtained by weighted superposition, so as to achieve comprehensive quality scoring of the face image.

[0048] S204: Determine whether the quality score is less than 10. If it is less than 10, directly proceed to step S211 and end; if it is greater than 10, proceed to step S205.

[0049] S205: Attribute analysis of captured face images (gender, age, facial features). Use the face detection engine to extract attributes and features of the face image, and obtain the following data: captured face image I, scene image B, capture time T, capture device ID, device name M, device type (Class A or Class B) D, gender S, image age N, and image face feature value F.

[0050] S206: Snapshot face clustering processing. Use the recognition engine function to cluster the image face features F. The clustering process is to calculate the similarity between the snapshot image Fm and the clustered personnel main images F0_0, F1_0, F2_0...Fn_0. If the similarity between Fm and Fx_0 is greater than or equal to the preset first threshold (e.g. 85), Fm is merged into the Fx_0 set. If the similarity between Fm and Fx_0 is greater than or equal to the preset second threshold (e.g. 65) but the similarity is less than the preset first threshold (e.g. 85), then three clustered personnel main images Fx_x, Fx_y, Fx_z and Fm are randomly selected from the cluster for comparison. If the similarity between two images is greater than the preset third threshold (e.g. 75), Fm is merged into the Fx_0 set. If the above conditions are not met, a new set Fm_0 is added.

[0051] S207: Determine whether to add a new aggregation subject. If yes, proceed to step S208, if not, proceed to step S209 for updating.

[0052] S208: Add the age, gender, appearance days, whether entering the store, last captured time, and first appearance time of the main picture.

[0053] S209: Update the main picture's age, gender, appearance days, whether it has entered the store, and the last captured time.

[0054] In a specific embodiment, the age is updated as follows: when the number of sets is less than or equal to 10, the average value of Nx_0, Nx_1, Nx_2, ... Nx_n is used as the set age; when the number of sets is greater than 10, Nx_0, Nx_1, Nx_2, ... Nx_n are used to calculate their arithmetic mean agv and residual error vi = Nx_i-agv (i = 0, 1, 2, ..., n), and the standard deviation σ is calculated according to the Bessel formula. If the residual error Nx_b (0 <= i <= n) of a certain measurement value Nx_i satisfies the following formula:

[0055] |vi|=|Nx_i-agv|>3σ

[0056] It is considered that Nx_b is a bad value containing a large error value and should be eliminated. Then the average value is calculated based on the eliminated age set value as the set age.

[0057] The specific method of updating gender is as follows: calculate based on the comparison of the set genders Sx_0, Sx_1, Sx_2, ... Sx_n, and the gender with a higher proportion is the clustered gender. If the same, the gender of the main image Sx_0 is used as the clustered gender.

[0058] In a specific embodiment, when merged into the personnel set Fx_0, it is determined whether the quality score of Fx_0 is less than Fm. If so, the main image is replaced with the features and quality score of Fm, otherwise it remains unchanged. At the same time, the last capture time, the number of days (deduplication based on the date), and whether the person has entered the store (the Class B camera appears in the set) are updated.

[0059] In a specific embodiment, when a new personnel set Fm_0 is added, the analysis data shows that the age of the main picture is the age of the set, the gender of the main picture is the gender of the set, the number of days of appearance is 1, whether the store has been entered (whether the main picture is a Class B camera, if so, the person has entered the store, otherwise, the person has not entered the store), the last capture time, and the first appearance date (the capture date of the current main picture).

[0060] S210: Data is stored. The data specifically includes personnel set identification, age, gender, number of days of appearance, whether to enter the store, last capture time, and first appearance date.

[0061] S211: End

[0062] Figure 3 A flowchart of passenger flow classification statistics based on face recognition according to a specific embodiment of the present application is shown. Figure 3 As shown, the specific process includes the following:

[0063] S301: Start.

[0064] S302: Data aggregation. The aggregated data content includes: personnel set identification, age, gender, appearance days, whether entering the store, last capture time, first appearance date; store sales content: the number of products sold on the day.

[0065] S303: Daily statistics [number of snapshots on the day, new people added on the day (gender ratio, age ratio, gender and age group ratio), new people entering the store on the day (gender ratio, age ratio, gender and age group ratio), store entry rate, purchase rate (number of goods sold / number of people entering the store)]. Based on the first appearance date being the same day, the total number of snapshots, the number of people gathered, the number of people entering the store (whether they entered the store is yes), store entry rate (new people entering the store / new people), and purchase rate (number of goods sold / number of people entering the store) are counted.

[0066] In a specific embodiment, the collective attributes of all personnel are counted: the number of males, the number of females, the number of people in the age group of 0-17 years old, the number of people in the age group of 18-35, the number of people in the age group of 36-60 years old, and the number of people in the age group of 61 and above. The age groups of males are (the number of 0-17, the number of 18-35, the number of 36-60, and the number of people above 61), and the age groups of females are (the number of 0-17, the number of 18-35, the number of 36-60, and the number of people above 61). The gender ratio, age group ratio, age group ratio of males, and age group ratio of females are derived.

[0067] In a specific embodiment, the attribute statistics of the set of people entering the store are: the number of males, the number of females, the number of people in the age group of 0-17 years old, the number of people in the age group of 18-35, the number of people in the age group of 36-60 years old, and the number of people in the age group of 61 and above. The age groups of males are (the number of 0-17, the number of 18-35, the number of 36-60, and the number of people above 61), and the age groups of females are (the number of 0-17, the number of 18-35, the number of 36-60, and the number of people above 61). The gender ratio, age group ratio, age group ratio of males, and age group ratio of females are derived.

[0068] S304: Statistics in the past week [number of snapshots in the past week, new people in the past week (gender ratio, age ratio, gender and age group ratio), new people entering the store in the past week (gender ratio, age ratio, gender and age group ratio), store entry rate, purchase rate (number of goods sold / number of people entering the store)]. Based on the first appearance date being a week, the statistical rules in S303 are repeated to obtain the statistical data of the past week.

[0069] S305: Statistics for the past month [number of snapshots for the past month, new people added for the past month (gender ratio, age ratio, ratio of gender and age group together), new people entering the store for the past month (gender ratio, age ratio, ratio of gender and age group together), store entry rate, purchase rate (number of goods sold / number of people entering the store)]. Based on the first appearance date being the most recent month, the statistical rules in S303 are repeated to obtain the statistical data for the past month.

[0070] In a specific embodiment, statistics are performed based on the number of days in which a group of people appear greater than or equal to 3 times in the most recent month as fixed customer flow, the number of fixed customer flow personnel, the number of fixed customer flow personnel entering the store, the fixed customer flow entry rate (the number of fixed customer flow personnel entering the store / the number of fixed customer flow personnel), the fixed customer flow purchase rate (the number of goods sold in the past month / the number of fixed customer flow personnel entering the store), and the gender ratio, age group ratio, male age group ratio, and female age group ratio are calculated at the same time.

[0071] S306: Data is stored.

[0072] S307: Data display. Display the obtained daily, weekly, monthly and fixed customer flow statistics, which can provide reference assistance for the customer flow of shops or stores or the effect of activities, and optimize the operation of shops or stores.

[0073] S308: End.

[0074] Continue to refer Figure 4 , Figure 4 The framework diagram of a passenger flow counting system based on face recognition according to an embodiment of the present invention is shown. The system specifically comprises a main graph set construction unit 401 for gathering people, a judgment unit 402, an attribute information maintenance unit 403 and a counting unit 404.

[0075] In a specific embodiment, the main picture set construction unit 401 for gathering people is configured to obtain a face image captured by an image acquisition device, extract face features using a face recognition engine, and construct a main picture set for gathering people of face images based on the face features; the judgment unit 402 is configured to respond to the situation that the similarity between a newly added face feature and a face feature in the main picture set for gathering people is greater than a first threshold, or the similarity is between the first threshold and the second threshold and the number of face features of face images randomly selected from the main picture set for gathering people whose similarity is greater than a third threshold is greater than half, merging the face image with the newly added face feature into the corresponding main picture set for gathering people, otherwise, adding the face image with the newly added face feature into the main picture set for gathering people; the attribute information maintenance unit 403 is configured to add or update the attribute information of the set, wherein the attribute information includes age, gender, store entry status, and appearance time; the statistical unit 404 is configured to associate the sales commodity records of the store, the main picture set for gathering people, and its attribute information, and obtain the customer flow statistics of the store.

[0076] Figure 5 FIG. 4 shows a functional structure diagram of a passenger flow counting system based on face recognition according to a specific embodiment of the present invention. Figure 5 As shown in the figure, Class A and Class B face capture cameras are installed inside and outside the store or shop, and the face data obtained by Class A and Class B capture devices are aggregated to the analysis server for analysis. Figure 1-Figure 3 The customer flow statistics analysis is finally performed and the statistical analysis results are displayed on the display screen, which can intuitively obtain the data information of customer flow and shops or stores.

[0077] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing an electronic device of an embodiment of the present application. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0078] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0079] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0080] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0081] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0084] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: obtains the face image captured by the image acquisition device, extracts the face features by using the face recognition engine, and constructs the main picture set of the face image based on the face features; in response to the similarity between the newly added face features and the face features in the main picture set of the gathered personnel being greater than the first threshold, or the similarity is between the first threshold and the second threshold and the number of face features of the face images randomly selected from the main picture set of the gathered personnel being greater than the third threshold exceeds half, the face image of the newly added face features is merged into the corresponding main picture set of the gathered personnel, otherwise, the face image of the newly added face features is added to the main picture set of the gathered personnel; the attribute information of the set is added or updated, wherein the attribute information includes age, gender, store entry status and appearance time; the sales commodity records of the associated store, the main picture set of the gathered personnel and its attribute information are obtained to obtain the customer flow statistics of the store.

[0085] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A passenger flow counting method based on face recognition, characterized in that: The following steps are involved: S1: Obtain a facial image captured by an image acquisition device, extract facial features using a facial recognition engine, and construct a main image set of people gathered in the facial image based on the facial features; S2: In response to the similarity between the newly added facial features and the facial features in the main image set of the gathered personnel being greater than a first threshold, or the similarity being between the first threshold and the second threshold and the number of facial features of the randomly selected facial images from the main image set of the gathered personnel being greater than a third threshold being greater than half, the facial image of the newly added facial features is merged into the corresponding main image set of the gathered personnel; otherwise, the facial image of the newly added facial features is added to the main image set of the gathered personnel; S3: adding or updating attribute information of the set, wherein the attribute information includes age, gender, store entry status and appearance time; and S4: associating the sales commodity records of the store, the main graph set of gathered people and their attribute information, and obtaining the customer flow statistics of the store; The specific method of updating age in step S3 is: In response to the number of face images in the subset of the main set of aggregated personnel images being less than a preset number, taking an average value as the age of the subset; In response to the fact that the number of facial images in a subset of the main set of aggregated personnel images is greater than a preset number, bad values ​​containing gross error values ​​are filtered out and the average value is taken as the age of the subset. The bad values ​​containing gross error values ​​are specifically: if the residual error of the measured value Nx_i satisfies |Vi|=|Nx_i-agv|<3σ, then the value is a bad value, where agv represents the arithmetic mean of the age of the subset, and σ is the standard deviation calculated using the Bessel formula.

2. The passenger flow counting method based on face recognition according to claim 1 is characterized in that: The image acquisition device includes an in-store capture device and an out-store capture device. The basis for judging the store entry status is: in response to the facial image being captured by the in-store capture device, the store entry status of the facial image is entered, otherwise it is not entered.

3. The passenger flow counting method based on face recognition according to claim 1 is characterized in that: The step S1 also includes obtaining the quality score of the face image and filtering the face images below the face image quality score threshold.

4. The passenger flow counting method based on face recognition according to claim 3 is characterized in that: The quality score of the face image is specifically expressed as the weighted superposition value of the pixel value, face ratio value, black and white gray pixel ratio value, eye quantity value, image clarity and face angle of the face image.

5. The passenger flow counting method based on face recognition according to claim 1 is characterized in that: The specific method of updating the gender in step S3 is: the gender with the highest proportion in the subset of the main picture set of the aggregated personnel is the gender of the subset.

6. The passenger flow counting method based on face recognition according to claim 1, characterized in that: The customer flow statistics include time-based basic statistics and set attribute statistics, wherein the basic statistics include counting the total number of captured facial images, counting the number of main image sets of gathered people, counting the number of subsets of people entering the store, the store entry rate and the purchase rate.

7. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 6 is implemented.

8. A passenger flow counting system based on face recognition, characterized in that: The system comprises: A main image collection construction unit for gathering people: configured to obtain a face image captured by an image acquisition device, extract face features using a face recognition engine, and construct a main image collection set of gathering people for the face image based on the face features; A judgment unit: configured to respond to the situation that the similarity between the newly added facial features and the facial features in the main picture set of the gathered personnel is greater than a first threshold, or the similarity is between the first threshold and the second threshold and the number of facial features of randomly selected facial images from the main picture set of the gathered personnel that are greater than a third threshold is greater than half, merging the facial image of the newly added facial features into the corresponding main picture set of the gathered personnel; otherwise, adding the facial image of the newly added facial features to the main picture set of the gathered personnel; An attribute information maintenance unit: configured to add or update attribute information of the set, wherein the attribute information includes age, gender, store entry status and appearance time; and Statistics unit: configured to associate the sales commodity records of the store, the main graph set of the gathered people and its attribute information, and obtain the customer flow statistics of the store; The specific method of updating age in the attribute information maintenance unit is: In response to the number of face images in the subset of the main set of aggregated personnel images being less than a preset number, taking an average value as the age of the subset; In response to the fact that the number of facial images in a subset of the main set of aggregated personnel images is greater than a preset number, bad values ​​containing gross error values ​​are filtered out and the average value is taken as the age of the subset. The bad values ​​containing gross error values ​​are specifically: if the residual error of the measured value Nx_i satisfies |Vi|=|Nx_i-agv|<3σ, then the value is a bad value, where agv represents the arithmetic mean of the age of the subset, and σ is the standard deviation calculated using the Bessel formula.

Images