A user behavior analysis system based on deep data mining and artificial intelligence

Abstract

The present invention relates to the technical field of user behavior analysis, and in particular to a user behavior analysis system based on deep data mining and artificial intelligence, the system comprising: a number-taking machine; a number-calling display screen; a hall camera module; a service terminal; a processing module, comprising: a behavior analysis unit for analyzing video stream data to obtain the behavior status of each person in the hall; a detection unit for detecting whether the behavior status of each person is the same as the preset behavior status in the behavior library; a to-be-verified number generation unit for obtaining the number-taking number of the person and using it as the to-be-verified number; a processing information acquisition unit for obtaining processing information of a preset number of number-taking numbers on the current number-calling display screen; an establishment determination unit for analyzing whether the behavior status corresponding to the to-be-verified number is established based on the processing information; and an instruction sending unit for sending a soothing instruction to the service terminal. The present application has the effect of responding to and soothing the behavior and emotions of the queueing personnel in a timely manner.

Description

Technical Field

[0001] The present invention relates to the technical field of user behavior analysis, and in particular to a user behavior analysis system based on deep data mining and artificial intelligence. Background Art

[0002] With the maturity of artificial intelligence technology, intelligent service systems have come into the spotlight. These systems can instantly identify customers upon arrival at bank branches and provide guidance. However, in terms of user-friendliness, for example, customers who need to obtain a number and then queue for bank transactions may become irritated and impatient due to various factors, leading to subsequent aggressive behavior or arguments with service staff. Bank systems lack the necessary oversight and timely guidance for these customers, so this system needs improvement. Summary of the Invention

[0003] In order to respond to and appease the behavior and emotions of queuers in a timely manner, this application provides a user behavior analysis system based on deep data mining and artificial intelligence.

[0004] The above-mentioned invention objective of this application is achieved through the following technical solutions:

[0005] A user behavior analysis system based on deep data mining and artificial intelligence, including:

[0006] A number-taking machine, which is used for people to take numbers. The number-taking machine is equipped with a number-taking camera module for capturing the facial image of the person taking the number, and is also used to associate the facial image with the number taken;

[0007] A call number display screen, wherein the call number display screen displays a sequence of a preset number of number pickup numbers, and the last number in the sequence on the call number display screen is the number pickup number currently being processed at the counter;

[0008] The lobby camera module is used to obtain video stream data in the waiting lobby;

[0009] A service terminal, which is provided to lobby service personnel;

[0010] Processing module, including:

[0011] Behavior analysis unit, used to analyze video stream data to obtain the behavior status of each person in the hall;

[0012] A detection unit is used to detect whether the behavior state of each person is the same as the preset behavior state in the behavior library;

[0013] A verification number generation unit is used to obtain the number of the person and use it as the verification number when the behavior state of the person is the same as the preset behavior state in the behavior library;

[0014] A processing information acquisition unit is used to obtain processing information of a preset number of number-picking numbers on the current call display screen, wherein the processing information includes the processing time and number type of each number-picking number;

[0015] a determination unit for analyzing whether the behavior state corresponding to the to-be-verified number is established based on the processed information;

[0016] The instruction sending unit is used to send a soothing instruction to the service terminal when the behavior state is established, and the soothing instruction carries the number to be verified and the corresponding face image.

[0017] By adopting the above technical solution, when people go to the bank to open accounts, deposit, loan, pay and settle other service activities, they first take a number through the number-taking machine. The number-taking camera module on the number-taking machine obtains the person's face image and associates it with the number taken by the person. Then the person can go to the rest area in the hall or wait in the hall for the number to be called to go to the counter to handle the service. The number-taking display screen can view the number of the number that has been called and the number of the number being handled. The hall camera module captures the video stream data in the hall in real time and sends it to the processing module. The behavior analysis unit of the processing module The frequency flow data is analyzed to obtain the behavioral status of each person, such as patting the thigh with a mobile phone, hammering, sighing, yawning, etc. The detection unit detects whether the behavioral status of each person is the preset behavioral status in the behavior library. At the same time, the verification number generation unit obtains the number of the person, and then analyzes whether the behavioral status corresponding to the number is established by processing the information. If it is established, it indicates that the person corresponding to the number is impatient, irritable, angry, etc., and a soothing instruction is sent to the service terminal. The service personnel equipped at the service terminal soothe the corresponding person after receiving the soothing instruction.

[0018] In a preferred example, the present application can be further configured as follows: the behavior analysis unit includes:

[0019] The current frame acquisition subunit is used to acquire the current frame image according to a preset period;

[0020] The sub-video stream acquisition sub-unit is used to acquire a preset number of video frame images before the current frame image as sub-video stream data based on the current frame image;

[0021] The analysis and reasoning sub-unit is used to input the data of each sub-video stream into a pre-trained analysis model and infer the behavior state.

[0022] By adopting the above technical solution, part of the sub-video stream data is intercepted from the video stream data at regular intervals for analysis. On the one hand, it can reduce the computing pressure, and on the other hand, it can ensure the continuity of user actions. The analysis is performed through model reasoning. As the number of training samples increases, the reasoning becomes more accurate.

[0023] In a preferred example, the present application can be further configured as follows: the preset period is 1 minute.

[0024] By adopting the above technical solution, part of the sub-video stream data is intercepted from the video stream data every 1 minute for analysis, which is more in line with reality and will not fail to capture the behavior of people.

[0025] In a preferred example, the present application can be further configured to: obtain the number of the person who took the number and use it as the number to be verified, including:

[0026] The corresponding number is matched according to the person's face image as the number to be verified.

[0027] By adopting the above technical solution, the facial image marked as the behavioral state in the image in the sub-video stream data is compared with the facial image when the number is initially taken, and then the corresponding number can be matched.

[0028] In a preferred example, the present application may be further configured as follows: analyzing whether the behavior state corresponding to the to-be-verified number is established according to the processing information includes:

[0029] Obtaining behavioral reasons based on processing information analysis, wherein the behavioral reasons include processing speed reasons, queue jumping reasons, and no reasons;

[0030] In a case where the behavior reason is a processing speed reason and / or a queue-jumping reason, it is determined that the behavior state corresponding to the pending number is established.

[0031] By adopting the above technical solution, if it is established, it means that the behavior status of the personnel is indeed related to the reason for queue jumping or the processing speed, then the subsequent processing will continue.

[0032] In a preferred example, the present application may be further configured as follows: analyzing whether the behavior state corresponding to the to-be-verified number is established according to the processing information further includes:

[0033] When the action reason is no reason, it is determined that the action state corresponding to the to-be-verified number is not established.

[0034] By adopting the above technical solution, if the condition is not met, it indicates that the behavior of the personnel is most likely not related to the reason for queue jumping or the processing speed.

[0035] In a preferred example, the present application may be further configured to include:

[0036] The strategy generating unit matches a corresponding soothing strategy according to the behavior cause when the behavior cause is a processing speed reason and / or a queue jumping reason.

[0037] By adopting the above technical solutions, corresponding soothing strategies are matched to guide the soothing work of service personnel so as to better fit the actual emotions of the personnel.

[0038] In a preferred example, the present application can be further configured as follows: the preset number is set to 5.

[0039] In summary, this application includes at least one of the following beneficial technical effects:

[0040] 1. When a person goes to a bank to open an account, make a deposit, take a loan, make a payment, or settle other services, he or she first takes a number from a number taking machine. The number taking camera module on the number taking machine obtains the person's face image and associates it with the number taken by the person. Then the person can go to the rest area in the hall or wait in the hall for the number to be called to go to the counter for service. The number taking display screen can view the number taking numbers that have been called and the number taking numbers currently being processed. The hall camera module captures the video stream data in the hall in real time and sends it to the processing module. The behavior analysis unit of the processing module processes the video stream data. The behavioral state of each person is analyzed, such as patting the thigh with a mobile phone, hammering, sighing, yawning, etc. The detection unit detects whether the behavioral state of each person is a preset behavioral state in the behavior library. If it is the same, the verification number generation unit obtains the number of the person, and then analyzes whether the behavioral state corresponding to the number is established by processing the information. If it is established, it indicates that the person corresponding to the number is impatient, irritable, angry, etc., and a soothing instruction is sent to the service terminal. The service personnel equipped at the service terminal soothe the corresponding person after receiving the soothing instruction;

[0041] 2. Intercepting sub-streams from the video stream at regular intervals for analysis can reduce computing pressure while ensuring the continuity of user actions. The analysis is performed through model inference. As the number of training samples increases, the inference becomes more accurate.

[0042] 3. If the result is established, it indicates that the person’s behavior is indeed related to the reason for queue jumping or processing speed, and subsequent processing will continue. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram showing the connections between modules of a user behavior analysis system based on deep data mining and artificial intelligence in one embodiment of the present application;

[0044] Figure 2This is a schematic diagram of the connections between modules of a user behavior analysis system based on deep data mining and artificial intelligence in another embodiment of the present application. DETAILED DESCRIPTION

[0045] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0046] It should be noted that the terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure.

[0047] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0048] Figure 1 FIG. 1 is a schematic diagram showing the connection of modules of a user behavior analysis system based on deep data mining and artificial intelligence in one embodiment of the present application. Figure 1 As shown, the user behavior analysis system based on deep data mining and artificial intelligence includes a number-taking machine for personnel to take numbers, a number-calling display screen, a hall camera module, a service terminal and a processing module for hall service personnel. The number-taking machine, the number-calling display screen, the hall camera module for obtaining video stream data in the waiting hall, the service terminal and the processing module are communicated with each other. Specifically, wireless communication is preferably used, such as LoRa technology, [WiFi / IEEE 802.11] protocol, [ZigBee / 802.15.4] protocol, [Thread / IEEE 802.15.4], [Z-Wave] protocol, etc. to achieve communication.

[0049] The number-taking machine is equipped with a number-taking camera module, which is used to capture the facial image of the person taking the number. For example, when a person arrives at the number-taking machine and enters information to take a number, the number-taking machine captures the facial image of the person in front of the display screen when the number is issued. This means that the facial image of the person taking the number is captured, and the number-taking machine then associates the facial image with the number taken by the person.

[0050] The call number display screen is hung high in the hall, and can be installed on the main beam of the hall to realize the public display function. The call number display screen displays a sequence of a preset number of number pickup numbers. In one embodiment, the preset number is set to 5, that is, the call number display screen displays 5 number pickup numbers according to the sequence of the bank's call number system, and the first four number pickup numbers are the number pickup numbers of people who have been processed, and the last number in the displayed sequence is the number pickup number of people who are currently receiving service at the counter;

[0051] The hall camera module is installed at a high position on a wall in the hall, so as to be able to capture the largest range in the hall. The hall camera module captures the video stream data in the hall and sends it to the processing module;

[0052] The processing module includes: a behavior analysis unit, a detection unit, a verification number generation unit, a processing information acquisition unit, a determination unit and an instruction sending unit. The behavior analysis unit is used to analyze the video stream data to obtain the behavior status of each person in the hall; specifically, refer to Figure 2 ,The behavior analysis unit includes : a current frame acquisition subunit, a sub-video stream acquisition subunit and an ,analysis and reasoning subunit;

[0053] The current frame acquisition subunit is configured to acquire the current frame image at a preset interval. The aforementioned hall camera module continuously transmits captured video stream data to the processing module, and the current frame acquisition subunit acquires the current frame image at a preset interval, such as once every minute. The current frame image refers to the frame image corresponding to the current timestamp. The sub-video stream acquisition subunit then acquires the video frame images preceding the current frame image by a preset number of frames as sub-video stream data based on the current frame image. The preset number of frames is pre-set, for example, 20 frames.

[0054] The analysis and reasoning sub-unit then inputs each sub-video stream data into a pre-trained analysis model to infer the behavior state. Specifically, the analysis model is trained in the following way:

[0055] Each image sample in the image sample training set is annotated to mark the behavioral state of each image sample, and the behavioral state is associated with all or part of the information in the image sample; and a neural network is trained using the annotated image sample training set to obtain an analysis model. Each image sample includes multiple frames of images, and the behavioral state can be slapping the thigh with a mobile phone, hammering, sighing, yawning, etc. The sample labeling is achieved through manual labeling. For example, a video clip reflecting the action of slapping the thigh with a mobile phone is intercepted from the video stream data as an image sample, and then the behavioral state of slapping the thigh with a mobile phone is labeled; a video clip reflecting the action of hammering is intercepted from the video stream data as an image sample, and then the behavioral state of hammering is labeled; and a video clip of a normal sitting posture without any behavior is intercepted from the video stream data as an image sample, and then the no behavior state is labeled; and the corresponding analysis model is obtained through training using these labeled sample data.

[0056] The detection unit is used to detect whether the behavior state of each person is the same as the preset behavior state in the behavior library; specifically, the behavior library is a pre-stored library for storing preset behavior states, and the preset behavior states stored in the behavior library may include slapping the thigh with a mobile phone, hammering, sighing, yawning, etc. The preset behavior state reflects the person's dissatisfaction, irritability, and impatience behavior and mentality. After the above-mentioned analysis model analyzes the behavior state in the sub-video stream data, it is compared with the preset behavior state in the behavior library. If the behavior state in the sub-video stream data can be matched in the behavior library, it indicates that the person may have dissatisfaction, irritability, and impatience behavior and mentality;

[0057] The verification number generation unit is used to obtain the number of the person and use it as the verification number when the behavior state of the person is the same as the preset behavior state in the behavior library; specifically, the corresponding number is matched according to the face image of the person as the verification number.

[0058] The processing information acquisition unit is used to obtain the processing information of a preset number of number-picking numbers on the current call display screen, and the processing information includes the processing time and number type of each number-picking number; continuing with the above example, if 5 number-picking numbers are displayed on the call display screen, the processing information acquisition unit obtains the processing time and number type of the 5 number-picking numbers. The processing time can be obtained by counting the adjacent intervals between calls, and the number type is usually divided into a variety of different business labels. For example, taking the number type of Bank of China as an example, N is the most basic personal business, K is corporate business, C is a VIP customer, B is a financial management customer, A is the highest-level private banking customer, and R is the number used by the lobby manager's card. Among them, C is a VIP customer who has assets of 200,000. B is a financial management client with assets of 2 million and above, and A is a private banking client with assets of approximately 7 million and above, which is the highest level. For example, if a number is C001, it means that the person is a VIP client with assets of 200,000 and above, and can be directly placed first in the queue. Correspondingly, numbers starting with N will be delayed by one position. N, K, C, B, A, and R all have different priorities. Generally speaking, N has the lowest priority, and the numbers of people with higher priorities will cause the numbers of people with lower priorities to be delayed by one position in the queue.

[0059] The establishment determination unit is used to analyze whether the behavior state corresponding to the to-be-verified number is established based on the processing information; specifically, analyzing whether the behavior state corresponding to the to-be-verified number is established based on the processing information includes:

[0060] The reasons for the behavior are obtained based on the analysis of the processing information. The reasons for the behavior include processing speed reasons, queue jumping reasons, and no reasons.

[0061] When the behavior reason is due to processing speed and / or queue jumping, it is determined that the behavior state corresponding to the pending number is established.

[0062] Analyze whether the behavior state corresponding to the pending number is established based on the processed information, and also include:

[0063] When the action reason is no reason, it is determined that the action state corresponding to the pending number is not established.

[0064] The behavior reason obtained from the processing information analysis is determined by judging the processing time and number type of the number. For example, if the processing time for five numbers exceeds a preset time, such as 90 minutes, it is determined to be a processing speed reason. If at least two of the five numbers have a higher priority than the number corresponding to the facial image, it is determined to be a queue jumping reason. If both are present, the behavior reason is determined to be processing speed and queue jumping. If neither is present, it is determined to be no reason.

[0065] If it is established, it means that the personnel's behavior is indeed related to the reason for queue jumping or processing speed, and then the subsequent processing will continue.

[0066] When the behavior state is established, the instruction sending unit sends a soothing instruction to the service terminal, where the soothing instruction carries the number to be verified and the corresponding face image.

[0067] In one embodiment, the processing module further includes a strategy generation unit. When the cause of the behavior is processing speed and / or queue jumping, the strategy generation unit matches the corresponding soothing strategy based on the cause of the behavior. Specifically, the system pre-stores a mapping table of behavior causes and soothing strategies. The corresponding soothing strategy can be directly matched based on the cause of the behavior. The soothing strategy can include soothing tone, phrases, expressions, and other information, providing guidance to service personnel that better aligns with their actual emotions.

[0068] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0070] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0071] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0072] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0073] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. User behavior analysis system based on deep data mining and artificial intelligence, characterized by: include: A number-taking machine, which is used for people to take numbers. The number-taking machine is equipped with a number-taking camera module for capturing the facial image of the person taking the number, and is also used to associate the facial image with the number taken; A call number display screen, wherein the call number display screen displays a sequence of a preset number of number pickup numbers, and the last number in the sequence on the call number display screen is the number pickup number currently being processed at the counter; The lobby camera module is used to obtain video stream data in the waiting lobby; A service terminal, which is provided to lobby service personnel; Processing module, including: Behavior analysis unit, used to analyze video stream data to obtain the behavior status of each person in the hall; A detection unit is used to detect whether the behavior state of each person is the same as the preset behavior state in the behavior library; A verification number generation unit is used to obtain the number of the person and use it as the verification number when the behavior state of the person is the same as the preset behavior state in the behavior library; A processing information acquisition unit is used to obtain processing information of a preset number of number-picking numbers on the current call display screen, wherein the processing information includes the processing time and number type of each number-picking number; a determination unit for analyzing whether the behavior state corresponding to the to-be-verified number is established based on the processed information; an instruction sending unit, configured to send a soothing instruction to the service terminal when the behavior state is established, wherein the soothing instruction carries the number to be verified and the corresponding facial image; a strategy generating unit, which matches a corresponding soothing strategy according to the cause of the behavior when the cause of the behavior is a processing speed reason and / or a queue jumping reason; The soothing strategy includes soothing tone, sentences and facial expressions.

2. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 1, characterized in that: The behavior analysis unit includes: The current frame acquisition subunit is used to acquire the current frame image according to a preset period; The sub-video stream acquisition sub-unit is used to acquire a preset number of video frame images before the current frame image as sub-video stream data based on the current frame image; The analysis and reasoning sub-unit is used to input the data of each sub-video stream into a pre-trained analysis model and infer the behavior state.

3. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 2, characterized in that: The preset period is 1 minute.

4. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 1, characterized in that: Obtain the person's number and use it as the verification number, including: The corresponding number is matched according to the person's face image as the number to be verified.

5. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 4, characterized in that: The analyzing, based on the processing information, whether the behavior state corresponding to the to-be-verified number is established includes: Obtaining behavioral reasons based on processing information analysis, wherein the behavioral reasons include processing speed reasons, queue jumping reasons, and no reasons; In a case where the behavior reason is a processing speed reason and / or a queue-jumping reason, it is determined that the behavior state corresponding to the pending number is established.

6. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 1, characterized in that: The analyzing, based on the processing information, whether the behavior state corresponding to the to-be-verified number is established further includes: When the action reason is no reason, it is determined that the action state corresponding to the to-be-verified number is not established.

7. The user behavior analysis system based on deep data mining and artificial intelligence according to claim 3, characterized in that: The preset number is set to 5.

Images