Online customer service allocation method and system

Abstract

The present invention proposes an online customer service allocation method and system. The online customer service allocation system includes a local server, an identity recognition module, a monitoring camera, a voice collection module, a real-time monitoring module and a cloud customer service server. The voice collection module includes a plurality of voice collection units, and each shelf is provided with a voice collection unit. The voice collection unit is used to collect sound signals emitted by customers close to the shelf. The real-time monitoring module includes a plurality of real-time monitoring units. Each shelf area is provided with a real-time monitoring unit. The real-time monitoring unit is used to collect temperature data, heartbeat data and attention data of customers in the monitoring area. The cloud customer service server allocates customer service according to the analysis results of the local server. In the present invention, customer service is allocated in a targeted manner according to the different states of the customer to better meet customer needs. In addition, under normal conditions, the monitoring data related to the customer is deleted in a timely manner to avoid privacy risks.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent customer service allocation, and in particular to an online customer service allocation method and system. Background Art

[0002] Unmanned supermarkets use advanced technologies such as the Internet of Things and artificial intelligence to realize self-service checkout and intelligent monitoring, greatly reducing labor costs. At the same time, through precise product management and inventory control, waste and loss are reduced, further improving profitability. As an innovative retail model, unmanned supermarkets meet the fast-paced lifestyle of modern people with their convenience and efficiency. They are gradually being accepted and recognized by consumers, and their application scenarios are also constantly expanding. Unmanned supermarkets are mainly operated through a cloud-attended customer service system, which serves customers through real-time monitoring. However, the cloud-attended customer service system still faces some drawbacks and challenges:

[0003] 1) A large amount of consumer data is collected through various sensors and cameras. If this data is improperly used or leaked, it may infringe on consumers' privacy rights and bring security risks;

[0004] 2) Customer service staff are easily fatigued and are unable to provide timely assistance when consumers encounter problems. Summary of the invention

[0005] In order to solve the technical problems of privacy risks and untimely service provision in the cloud-attended customer service system in the prior art, the online customer service allocation system proposed in the present invention includes a local server, an identity recognition module, a monitoring camera, a voice collection module, a real-time monitoring module and a cloud-based customer service server, wherein the local server is respectively connected to the identity recognition module, the monitoring camera, the voice collection module, the real-time monitoring module and the cloud-based customer service server for communication;

[0006] The voice collection module includes a plurality of voice collection units, one voice collection unit is provided for each shelf, and the voice collection unit is used to collect sound signals emitted by customers approaching the shelf;

[0007] The real-time monitoring module includes a plurality of real-time monitoring units, one real-time monitoring unit is correspondingly arranged in each shelf area, and the real-time monitoring unit is used to collect the body temperature data, heart rate data and attention data of customers in the monitoring area;

[0008] The cloud customer service server allocates customer service according to the analysis results of the local server.

[0009] Preferably, the identity recognition module includes mounting plates arranged at both ends of the entrance, and the mounting plates are provided with a control device, an upper camera, a first digital pickup and a lower camera in sequence from top to bottom.

[0010] Preferably, the shooting range of the upper camera can be adjusted within the range of ±45°, the shooting angle of the lower camera is 45° upward, the first digital microphone is used to collect the customer's voice signal, and transmit the sound signal to the local server through the control device, and the local server can extract the customer's breathing sound characteristics based on the sound signal. The control device includes an infrared sensor, a voice prompt module and a processor.

[0011] Preferably, the voice collection unit includes a first processing chip and a plurality of second digital pickups, the second digital pickups are arranged on the surface of the shelf to collect sound signals, the first processing chip is arranged inside the shelf to receive the signals of the second digital pickups and transmit them to the local server, the local server extracts the customer's breathing sound characteristics according to the sound signals, or determines whether the customer sends a signal for help according to the sound signals.

[0012] Preferably, the specific process of the local server extracting respiratory sound features based on sound signals is to perform high-pass filtering on the original sound signals to remove environmental noise in the original sound signals to obtain preprocessed sound signals, perform waveform time domain analysis on the preprocessed sound signals to obtain respiratory frequency and expiratory phase ratio, divide the preprocessed sound signals into multiple single-cycle sound signals, perform Fourier transform on the single-cycle sound signals to obtain corresponding two-dimensional spectrograms, obtain a spectrogram data set, correspond all the two-dimensional spectrograms in the spectrogram data set according to pixels, average the grayscale values ​​of corresponding pixels in all the two-dimensional spectrograms as the grayscale value of the pixels in the synthesized two-dimensional spectrogram, obtain a final synthesized two-dimensional spectrogram, and use the synthesized two-dimensional spectrogram, respiratory frequency and expiratory phase ratio as respiratory sound features.

[0013] Preferably, the real-time monitoring unit includes a rotating workbench, a second processing chip is arranged on the rotating workbench, an equipment mounting rod is arranged in the center of the rotating workbench, an infrared thermal imager, a biological radar and an identification camera are arranged on the equipment mounting rod, and the second processing chip is wirelessly connected to the infrared thermal imager, the biological radar and the identification camera respectively, and the second processing chip transmits the data of the infrared thermal imager, the biological radar and the identification camera to a local server.

[0014] Preferably, the bioradar is a K-band millimeter-wave radar.

[0015] Preferably, the cloud-based customer service server allocates customer service based on the analysis results of the local server, specifically including the local server determining the customer status based on data from the identity recognition module, the voice collection module and the real-time monitoring unit, wherein the customer status includes whether the customer is at security risk, the customer needs real-time help and the customer needs appropriate help, and customer service is allocated based on the customer status.

[0016] The online customer service allocation method proposed in the present invention is based on the above-mentioned online customer service allocation system, and is characterized in that the online customer service allocation method comprises the following steps:

[0017] S1, the identity recognition module collects the customer's facial image and voice signal;

[0018] S2, the local server determines whether the facial features of the customer can be obtained based on the facial image, if so, enter S3, otherwise, enter S8;

[0019] S3, the local server obtains the customer's facial features according to the facial image, obtains the customer's breathing sound features according to the sound signal, and stores the customer's facial features and breathing sound features;

[0020] S4, obtaining the estimated age of the customer based on the facial features, and determining whether the estimated age is greater than an age threshold, if so, proceeding to S5, otherwise, proceeding to S6;

[0021] S5. The customer status is determined to be that the customer needs real-time help, and the cloud customer service server assigns real-time customer service to the customer, and then proceeds to S7;

[0022] S6. Determine that the customer is in need of appropriate help, obtain a help signal from the customer through voice through the voice collection module, and the cloud customer service server assigns a customer service representative to the customer according to the help signal;

[0023] S7, monitoring whether the customer has theft risk through the real-time monitoring module, if yes, determining the customer's breathing sound characteristics through the voice collection module, determining the corresponding facial features according to the breathing sound characteristics, transmitting the customer's facial features and breathing sound characteristics to the cloud customer service server, and entering S8, otherwise, entering S9;

[0024] S8. The customer status is judged to be a security risk, and the cloud customer service server assigns a security customer service to the customer, ends the customer service assignment, and does not enter S9;

[0025] S9. When the customer leaves, the local server deletes the monitoring data related to the customer based on the customer's facial features and breathing sound features.

[0026] Preferably, in S7, when the body temperature data and heart rate data acquired by the real-time monitoring unit are beyond the normal range and the attention data is less than the set threshold, the customer is at risk of theft, and the monitoring camera determines the voice collection unit corresponding to the customer based on the location of the real-time monitoring unit, and the local server extracts the customer's breathing sound characteristics based on the sound signal collected by the voice collection unit.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] Customer service can be assigned in a targeted manner according to the different status of customers to better meet customer needs. In addition, under normal circumstances, customer-related monitoring data can be deleted in a timely manner to avoid privacy risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a structural diagram of the online customer service allocation system of the present invention;

[0030] Figure 2 It is a flow chart of the online customer service allocation method of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0032] like Figure 1 As shown, the online customer service allocation system proposed in the present invention includes a local server, an identity recognition module, a surveillance camera, a voice acquisition module, a real-time monitoring module and a cloud customer service server. The local server is respectively communicated with the identity recognition module, the surveillance camera, the voice acquisition module, the real-time monitoring module and the cloud customer service server.

[0033] The identity recognition module includes a mounting plate arranged at both ends of the entrance, and the mounting plate is provided with a control device, an upper camera, a first digital microphone and a lower camera in order from top to bottom. The shooting range of the upper camera can be adjusted within the range of ±45°, and the shooting angle of the lower camera is 45° upward. The upper camera and the lower camera cooperate with each other to ensure that the facial features of the customer are captured. The first digital microphone is used to collect the voice signal of the customer, and transmit the voice signal to the local server through the control device. The local server can extract the breathing sound characteristics of the customer according to the voice signal. The control device includes an infrared sensor, a voice prompt module and a processor. The processor detects whether a customer enters through the infrared sensor. When a customer enters, the voice prompt module reminds the customer to stay for 3 minutes so that the upper camera, the first digital microphone and the lower camera can collect signals. At the same time, the upper camera, the first digital microphone and the lower camera are notified to start working and transmit the collected signals to the local server.

[0034] Surveillance cameras are evenly distributed throughout the shopping area.

[0035] The voice collection module includes multiple voice collection units, and each shelf is provided with a voice collection unit. The voice collection unit is used to collect sound signals emitted by customers approaching the shelf. The voice collection unit includes a first processing chip and multiple second digital pickups. The second digital pickups are arranged on the surface of the shelf to collect sound signals. The first processing chip is arranged inside the shelf to receive the signals of the second digital pickups and transmit them to the local server. The local server extracts the customer's breathing sound characteristics according to the sound signal, or determines whether the customer sends a signal for help according to the sound signal.

[0036] The specific process of the local server extracting respiratory sound features based on the sound signal is to perform high-pass filtering on the original sound signal to remove environmental noise from the original sound signal to obtain a preprocessed sound signal, perform waveform time domain analysis on the preprocessed sound signal to obtain the respiratory frequency and the proportion of the expiratory phase, divide the preprocessed sound signal into multiple single-cycle sound signals, and obtain the corresponding two-dimensional spectrogram of the single-cycle sound signal through Fourier transform to obtain a spectrogram data set, correspond all the two-dimensional spectrograms in the spectrogram data set according to pixels, calculate the average grayscale values ​​of the corresponding pixels in all the two-dimensional spectrograms as the grayscale value of the pixels in the synthetic two-dimensional spectrogram, obtain the final synthetic two-dimensional spectrogram, and use the synthetic two-dimensional spectrogram, respiratory frequency and expiratory phase proportion as respiratory sound features.

[0037] The real-time monitoring module includes multiple real-time monitoring units. Each shelf area is provided with a real-time monitoring unit. The real-time monitoring unit is used to collect the temperature data, heart rate data and attention data of customers in the monitoring area. The real-time monitoring unit includes a rotating workbench, on which a second processing chip is arranged. A device mounting rod is arranged in the center of the rotating workbench. An infrared thermal imager, a biological radar and an identification camera are arranged on the device mounting rod. The second processing chip is wirelessly connected with the infrared thermal imager, the biological radar and the identification camera respectively. The second processing chip transmits the data of the infrared thermal imager, the biological radar and the identification camera to the local server. The infrared thermal imager uses infrared waves to perform infrared imaging of the customer, and the customer's body temperature data can be obtained based on the infrared image. The biological radar is a millimeter wave radar in the K band. The biological radar uses 24GHz radar waves to scan the human body. The reflected radar waves contain the human heartbeat signal. The customer's heartbeat data can be obtained by analyzing the reflected radar waves. The functions of the recognition camera include: 1) locating the infrared thermal imager and bio-radar, and judging whether the infrared thermal imager and bio-radar are currently aimed at the customer by recognizing the camera's image; 2) obtaining attention data, identifying the camera to shoot the customer's shopping process, analyzing the captured video frame by frame, taking the frame in which the customer's head is facing the shelf as the target frame, calculating the proportion of the target frame in all the frames, and using this proportion as the attention data.

[0038] The cloud customer service server allocates customer service according to the analysis results of the local server. The local server determines the customer status based on the data of the identity recognition module, voice collection module and real-time monitoring unit. The customer status includes the customer being at risk of security, the customer needing real-time help and the customer needing appropriate help. Customer service is allocated according to the customer status. If the customer status is that the customer is at risk of security, the cloud customer service server will allocate security customer service to the customer. The security customer service will control the surveillance camera to take pictures of the customer, and combine the data of the identity recognition module, voice collection module and real-time monitoring unit to form complete identity recognition evidence. When the customer commits crimes such as theft, the police will be called in time to provide sufficient evidence for the police to handle the case. The method of judging whether the customer has security risks is for two situations: Situation 1: The customer originally has the intention of committing crimes such as theft, and hides his features by covering his face, wearing sunglasses and other means. At this time, the judgment method is based on the facial image obtained by the identity recognition module, and the facial features of the customer cannot be obtained; Situation 2: The customer suddenly intends to commit crimes such as theft during shopping. At this time, the judgment method is that the temperature data and heartbeat data obtained by the real-time monitoring unit exceed the normal range, and the attention data is less than the set threshold. If the customer status is that the customer needs real-time help, the cloud customer service server will assign real-time customer service to the customer. The real-time customer service will closely monitor the customer's entire shopping process and provide timely help to the customer. The method for judging whether a customer needs real-time help is to judge the customer's age based on the facial image obtained by the identity recognition module. If the age is greater than the set threshold, it means that the customer needs real-time help. If the customer status is that the customer needs appropriate help, the cloud customer service server will only assign a customer service to the customer after the customer sends a help signal through voice.

[0039] like Figure 2 As shown, based on the above online customer service allocation system, the online customer service allocation method includes the following steps:

[0040] S1. The identity recognition module collects the customer's facial image and voice signal.

[0041] S2. The local server determines whether the facial features of the customer can be obtained based on the facial image. If so, it proceeds to S3; otherwise, it proceeds to S8.

[0042] S3. The local server obtains the customer's facial features based on the facial image, obtains the customer's breathing sound features based on the sound signal, and stores the customer's facial features and breathing sound features.

[0043] S4. Obtain the estimated age of the customer based on the facial features, and determine whether the estimated age is greater than the age threshold. If so, proceed to S5; otherwise, proceed to S6.

[0044] S5. The customer status is determined to be that the customer needs real-time help, and the cloud customer service server assigns real-time customer service to the customer, and then enters S7.

[0045] S6. The customer status is determined to be that the customer needs appropriate help, and a help signal sent by the customer through voice is obtained through the voice collection module. The cloud customer service server assigns customer service to the customer according to the help signal.

[0046] S7, monitor whether the customer is at risk of theft through the real-time monitoring module. If so, determine the customer's breathing sound characteristics through the voice collection module, determine the corresponding facial features based on the breathing sound characteristics, transmit the customer's facial features and breathing sound characteristics to the cloud customer service server, and enter S8. Otherwise, enter S9. Specifically, when the body temperature data and heart rate data obtained by the real-time monitoring unit exceed the normal range, and the attention data is less than the set threshold, the customer is at risk of theft. The monitoring camera determines the voice collection unit corresponding to the customer based on the location of the real-time monitoring unit, and the local server extracts the customer's breathing sound characteristics based on the sound signal collected by the voice collection unit.

[0047] S8. The customer status is judged to be a security risk for the customer. The cloud customer service server assigns a secure customer service to the customer and ends the customer service assignment without entering S9.

[0048] S9. When the customer leaves, the local server deletes the monitoring data related to the customer based on the customer's facial features and breathing sound features.

[0049] The above disclosure is only the preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the present invention. It should be pointed out that for those skilled in the art, any equivalent changes made to the scheme of the present invention without departing from the design structure and principle of the present invention are considered to be within the protection scope of the present invention.

Claims

1. An online customer service allocation method, based on an online customer service allocation system, characterized in that ,The online customer service allocation system includes a local server, an identity recognition module, a surveillance camera, a voice collection module, a real-time monitoring module and a cloud customer service server, wherein the local server is respectively communicatively connected with the identity recognition module, the surveillance camera, the voice collection module, the real-time monitoring module and the cloud customer service server; The voice collection module includes a plurality of voice collection units, one voice collection unit is provided for each shelf, and the voice collection unit is used to collect sound signals emitted by customers approaching the shelf; The real-time monitoring module includes a plurality of real-time monitoring units, one real-time monitoring unit is correspondingly arranged in each shelf area, and the real-time monitoring unit is used to collect the body temperature data, heart rate data and attention data of customers in the monitoring area; The cloud customer service server performs customer service allocation according to the analysis results of the local server; The online customer service allocation method comprises the following steps: S1, the identity recognition module collects the customer's facial image and voice signal; S2, the local server determines whether the facial features of the customer can be obtained based on the facial image, if so, enter S3, otherwise, enter S8; S3, the local server obtains the customer's facial features according to the facial image, obtains the customer's breathing sound features according to the sound signal, and stores the customer's facial features and breathing sound features; S4, obtaining the estimated age of the customer based on the facial features, and determining whether the estimated age is greater than an age threshold, if so, proceeding to S5, otherwise, proceeding to S6; S5. The customer status is determined to be that the customer needs real-time help, and the cloud customer service server assigns real-time customer service to the customer, and then proceeds to S7; S6. Determine that the customer is in need of appropriate help, obtain a help signal from the customer through voice through the voice collection module, and the cloud customer service server assigns a customer service representative to the customer according to the help signal; S7, monitoring whether the customer has theft risk through the real-time monitoring module, if yes, determining the customer's breathing sound characteristics through the voice collection module, determining the corresponding facial features according to the breathing sound characteristics, transmitting the customer's facial features and breathing sound characteristics to the cloud customer service server, and entering S8, otherwise, entering S9; S8. The customer status is judged to be a security risk, and the cloud customer service server assigns a security customer service to the customer, ends the customer service assignment, and does not enter S9; S9. When the customer leaves, the local server deletes the monitoring data related to the customer based on the customer's facial features and breathing sound features.

2. The online customer service allocation method according to claim 1, characterized in that: The identity recognition module comprises mounting plates arranged at both ends of the entrance, and the mounting plates are provided with a control device, an upper camera, a first digital pickup and a lower camera in sequence from top to bottom.

3. The online customer service allocation method according to claim 2, characterized in that: The shooting range of the upper camera is adjusted within the range of ±45°, the shooting angle of the lower camera is 45° upwards, the first digital microphone is used to collect the customer's voice signal, and the sound signal is transmitted to the local server through the control device. The local server extracts the customer's breathing sound characteristics based on the sound signal. The control device includes an infrared sensor, a voice prompt module and a processor.

4. The online customer service allocation method according to claim 1, characterized in that: The voice collection unit includes a first processing chip and multiple second digital microphones. The second digital microphones are arranged on the surface of the shelf to collect sound signals. The first processing chip is arranged inside the shelf to receive the signals of the second digital microphones and transmit them to the local server. The local server extracts the customer's breathing sound characteristics based on the sound signals, or determines whether the customer sends a help signal based on the sound signals.

5. The online customer service allocation method according to claim 2 or claim 4, characterized in that: The specific process of the local server extracting respiratory sound features based on sound signals is as follows: performing high-pass filtering on the original sound signals to remove environmental noise from the original sound signals to obtain preprocessed sound signals, performing waveform time domain analysis on the preprocessed sound signals to obtain respiratory frequency and expiratory phase ratio, dividing the preprocessed sound signals into multiple single-cycle sound signals, performing Fourier transform on the single-cycle sound signals to obtain corresponding two-dimensional spectrograms, obtaining a spectrogram data set, corresponding all two-dimensional spectrograms in the spectrogram data set according to pixels, averaging the grayscale values ​​of corresponding pixels in all two-dimensional spectrograms as the grayscale value of pixels in the synthesized two-dimensional spectrogram, obtaining a final synthesized two-dimensional spectrogram, and using the synthesized two-dimensional spectrogram, respiratory frequency and expiratory phase ratio as respiratory sound features.

6. The online customer service allocation method according to claim 1, characterized in that: The real-time monitoring unit includes a rotating workbench, on which a second processing chip is arranged, and an equipment mounting rod is arranged in the center of the rotating workbench, on which an infrared thermal imager, a biological radar and an identification camera are arranged, and the second processing chip is wirelessly connected to the infrared thermal imager, the biological radar and the identification camera respectively, and the second processing chip transmits the data of the infrared thermal imager, the biological radar and the identification camera to a local server.

7. The online customer service allocation method according to claim 6, characterized in that: The biological radar is a K-band millimeter-wave radar.

8. The online customer service allocation method according to claim 1, characterized in that: The cloud-based customer service server allocates customer service based on the analysis results of the local server, specifically including the local server determining the customer status based on data from the identity recognition module, the voice collection module and the real-time monitoring unit, wherein the customer status includes whether the customer is at security risk, the customer needs real-time help and the customer needs appropriate help, and customer service is allocated based on the customer status.

9. The online customer service allocation method according to claim 1, characterized in that: In S7, when the body temperature data and heart rate data acquired by the real-time monitoring unit are beyond the normal range and the attention data is less than the set threshold, the customer is at risk of theft. The surveillance camera determines the voice collection unit corresponding to the customer based on the location of the real-time monitoring unit, and the local server extracts the customer's breathing sound characteristics based on the sound signal collected by the voice collection unit.

Images