Service method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a service method and device, equipment, a storage medium and a program product, and relates to the technical field of financial science and technology or artificial intelligence. The method comprises the following steps: collecting target customer information, wherein the target customer information comprises first face image data and first audio data of a target customer; performing facial feature extraction based on the first facial image data to obtain first facial feature information; performing audio feature extraction based on the first audio data to obtain first audio feature information; determining first state information according to the first facial feature information and the first audio feature information, wherein the first state information is used for indicating an emotional state of the target customer; under the condition that the first condition is met, first information is sent to the first terminal, the first information is used for indicating the first terminal to interact with the target client, and the first condition includes that the first state information meets the second condition. According to the service method provided by the invention, the satisfaction degree of bank services is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of financial technology or artificial intelligence, in particular to a service method, device, equipment, storage medium and program product. BACKGROUND

[0002] The existing bank outlet service mode is mainly a traditional manual service mode. A teller at a bank counter needs to observe the facial expression of a customer and listen to the tone of the customer to determine the state of the customer.

[0003] However, the teller needs to perform business operations and cannot continuously pay attention to the customer, which makes it difficult to discover potential problems or negative emotions of the customer in a timely manner, so that the needs of the customer cannot be met and the customer satisfaction of the bank is low. SUMMARY

[0004] The present application provides a service method, device, equipment, storage medium and product to improve the customer satisfaction of a bank.

[0005] In a first aspect, the present application provides a service method, comprising: collecting target customer information, the target customer information comprising first facial image data of a target customer and first audio data of the target customer; performing facial feature extraction based on the first facial image data to obtain corresponding first facial feature information; performing audio feature extraction based on the first audio data to obtain corresponding first audio feature information; determining first state information corresponding to the target customer according to the first facial feature information and the first audio feature information, the first state information being used to indicate the emotional state of the target customer; and sending first information to a first terminal in a case where a first condition is met, the first information being used to instruct the first terminal to interact with the target customer, and the first condition comprising that the first state information meets a second condition.

[0006] In a possible implementation manner, the first facial feature information comprises a first facial emotion score, the performing facial feature extraction based on the first facial image data to obtain corresponding first facial feature information comprises: performing facial feature extraction based on the first facial image data to obtain corresponding first facial emotion score; the first audio feature information comprises a first audio emotion score, the performing audio feature extraction based on the first audio data to obtain corresponding first audio feature information comprises: performing audio feature extraction based on the first audio data to obtain corresponding first audio emotion score; and the first state information comprises a first emotion score, the determining first state information corresponding to the target customer according to the first facial feature information and the first audio feature information comprises: determining the first emotion score corresponding to the target customer according to the first facial emotion score, a first weight corresponding to the first facial emotion score, the first audio emotion score and a second weight corresponding to the first audio emotion score.

[0007] In a possible implementation, the service method further includes: determining interaction instruction information according to the first audio data, the interaction instruction information including a service type corresponding to the target customer and / or a question title related to the service type; and the first information includes the interaction instruction information, and the first information is used to instruct the first terminal to interact with the target customer according to the interaction instruction information.

[0008] In a possible implementation, the first information further includes first state information.

[0009] In a possible implementation, before the first information is sent to the first terminal, the service method further includes: obtaining a current position of each intelligent terminal in the plurality of intelligent terminals; and determining the first terminal from the plurality of intelligent terminals according to the current position of each intelligent terminal and / or the first emotion score, the first terminal being an intelligent terminal in the plurality of intelligent terminals that has a minimum distance to a current position of the target customer.

[0010] In a possible implementation, before the first information is sent to the first terminal, the service method further includes: determining a priority of the target customer in the plurality of customers according to the first emotion score; and determining the first terminal corresponding to the target customer according to the priority of the target customer in the plurality of customers.

[0011] In a possible implementation, the first condition further includes: receiving a first operation from the first worker, the first operation being used to instruct the intelligent terminal to interact with the target customer.

[0012] In a possible implementation, the service method further includes: receiving third information from the first terminal, the third information indicating that the interaction of the first terminal with the target customer fails to solve a problem of the target customer.

[0013] In a possible implementation, the service method further includes: receiving third information from the first terminal, the third information indicating that the interaction of the first terminal with the target customer fails to solve a problem of the target customer.

[0014] In a possible implementation, the first information includes interaction instruction information, the interaction instruction information including a service type corresponding to the target customer and / or a question title related to the service type, and the interaction of the first terminal with the target customer according to the first information to obtain the first interaction result includes: interacting with the target customer according to the service type corresponding to the target customer and / or the question title related to the service type.

[0015] In one possible implementation, the first information further includes first state information. Interacting with the target customer according to the interaction instruction information to obtain a first interaction result includes: interacting with the target customer according to the interaction instruction information; collecting second customer information, which includes second facial image data and second audio data of the target customer; extracting facial features based on the second facial image data to obtain corresponding second facial feature information; extracting audio features based on the second audio data to obtain corresponding second audio feature information; determining second state information corresponding to the target customer based on the second facial feature information and the second audio feature information, the second state information indicating the target customer's emotional state; and determining the first interaction result based on the second state information, or the first state information and the second state information.

[0016] In one possible implementation, the service method further includes: if the first interaction result indicates that the target customer's problem is not resolved, sending third information to the first device, the third information indicating that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem.

[0017] In one possible implementation, the first terminal includes an indicator light, and the service method further includes: determining a corresponding first prompting mode among multiple prompting modes based on a first interaction result, wherein the first interaction result is one of unresolved problem, partially resolved problem, or resolved problem, and the multiple prompting modes correspond to different colors, different brightness, and / or different flashing frequencies; and performing prompting work through the indicator light in the first prompting mode.

[0018] Thirdly, this application provides a service device, comprising: a first transceiver module and a first processing module; the first transceiver module is used to collect target customer information, the target customer information including first facial image data and first audio data of the target customer; the first processing module is used to extract facial features based on the first facial image data to obtain corresponding first facial feature information; the first processing module is also used to extract audio features based on the first audio data to obtain corresponding first audio feature information; the first processing module is also used to determine first state information corresponding to the target customer based on the first facial feature information and the first audio feature information, the first state information being used to indicate the emotional state of the target customer; the first transceiver module is also used to send first information to a first terminal when a first condition is met, the first information being used to instruct the first terminal to interact with the target customer, the first condition including the first state information meeting a second condition.

[0019] Fourthly, this application provides a service apparatus, including: a second transceiver module and a second processing module; the second transceiver module is used to receive first information, the first information instructing a first terminal to interact with a target customer; the second processing module is used to interact with the target customer based on the first information to obtain a first interaction result.

[0020] Fifthly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor;

[0021] The memory stores the instructions that the computer executes;

[0022] The processor executes computer-executable instructions stored in memory to implement the method of either the first aspect or the method of either the second aspect.

[0023] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method of any one of the first aspects or the method of any one of the second aspects.

[0024] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects or the method of any one of the second aspects.

[0025] The service method provided in this application collects target customer information through a first device and determines the customer's first state information accordingly. This first state information indicates the customer's emotional state, using the device to replace subjective human judgment, achieving an objective and quantitative assessment of customer emotions, ensuring the timeliness and accuracy of service response, and improving bank satisfaction. On the other hand, it also allows interaction with customers through a first terminal, providing services and reducing the problem of tellers being unable to provide timely service due to busy schedules, further meeting customer needs and improving bank satisfaction. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] Figure 1 This application provides a schematic diagram of the architecture of a service system.

[0028] Figure 2 A flowchart illustrating a service method provided in an embodiment of this application;

[0029] Figure 3 A flowchart illustrating another service system provided in this application embodiment;

[0030] Figure 4 This is a schematic diagram of the structure of a first device provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of the structure of a first terminal provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of another first device provided in an embodiment of this application;

[0033] Figure 7 This is a schematic diagram of another first terminal provided in an embodiment of this application;

[0034] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0035] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0038] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0039] It should be noted that the service methods, apparatus, equipment, storage media and products provided in this application can be used in the fields of financial technology or artificial intelligence technology, or in any field other than financial technology or artificial intelligence technology. This application does not limit the application fields of the service methods, apparatus, equipment, storage media and products.

[0040] The specific application scenario for this application is a bank's intelligent service system. Figure 1 This is a schematic diagram of the architecture of a service system 100 provided in an embodiment of this application. The service system 100 includes a first device 101 and a first terminal 102, and optionally, a second terminal 103. The first device 101 may include a customer information collection module 1011 and a comprehensive control module 1012.

[0041] The first device 101 is used to acquire customer emotional state information through the customer information acquisition module 1011, and to determine whether the first terminal 102 needs to interact with the customer based on the customer emotional state information through the integrated control module 1012.

[0042] Optionally, the second terminal 103 can be used to instruct lobby staff to interact with customers.

[0043] In this embodiment of the application, the first device may be an electronic device including a customer information collection module and an integrated control module, the first terminal may be a mobile intelligent robot or other terminal, and the second terminal may be a smart tablet or a smartphone or other terminal.

[0044] The existing bank branch service model is mainly the traditional manual service model, in which bank tellers need to judge the customer's state by visually observing the customer's facial expressions and listening to the customer's tone of voice.

[0045] However, because tellers need to perform business operations, it is difficult for them to continuously monitor customer dynamics. This makes it difficult for tellers to discover potential problems or negative emotions of customers in a timely manner, resulting in unmet customer needs and low bank satisfaction.

[0046] The service method provided in this application is intended to solve the above-mentioned technical problems of the prior art.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] Figure 2 This is a flowchart illustrating a service method provided in an embodiment of this application, such as... Figure 2 As shown, the method includes:

[0049] S201, The first device collects target customer information of the target customer, which includes the target customer's first facial image data and first audio data.

[0050] In one possible implementation, such as Figure 3 As shown, the first device may include an information acquisition module, which may include a shooting unit. The shooting unit can be used to acquire facial image data of the target customer at a certain moment or time period, thereby obtaining the first image data.

[0051] In one possible implementation, the information acquisition module may include a voice recording unit, which can be used to acquire audio data of the target customer at a certain moment or time period, thereby obtaining the first audio data.

[0052] In this embodiment, the first device acquires target customer information to obtain the target customer's emotional state. Therefore, it is understood that, in addition to the first facial image data and the first audio data, the target customer information may also include other information about the target customer that can indicate their emotional state; that is, the first device may also acquire other information about the target customer. For example, when someone is emotionally agitated, their body temperature will rise. Correspondingly, the information acquisition module may also include a temperature detection unit to acquire the target customer's first temperature information.

[0053] In one possible implementation, the service method provided in this application embodiment can be applied to a bank. The shooting unit, voice recording unit, and other units of the first device used to obtain target customer information can be deployed at the bank counter window to collect information from customers handling counter business.

[0054] In one possible implementation, the first device may include a human-computer interaction module, which may include a touch screen, a microphone, and a speaker. The human-computer interaction module may be set at the bank counter window for human-computer interaction with customers or tellers handling counter business.

[0055] S202. The first device extracts facial features based on the first facial image data to obtain corresponding first facial feature information, and extracts audio features based on the first audio data to obtain corresponding first audio feature information.

[0056] In some embodiments, the first facial feature information may include a first facial emotion score. The first device extracts facial features based on the first facial image data to obtain the first facial features. Then, the first facial features are compared with the baseline features of the facial image to obtain the corresponding first facial emotion score, which can be used to indicate the facial emotional state of the target customer.

[0057] For example, the first device can extract features from the first facial image data through a processing module, and then compare the obtained first facial feature information with the facial image pleasure benchmark features to obtain a corresponding first pleasure score. This first pleasure score can be used to indicate the level of pleasure of the target customer. The more pleasure the target customer is experiencing, the higher the pleasure score value; the less pleasure the target customer is experiencing, the lower the pleasure score value.

[0058] In one possible implementation, the first device can extract facial features from the facial image data using a deep convolutional neural network to obtain the corresponding facial feature information.

[0059] For example, the first device can extract features from the first facial image data using a convolutional neural network, such as the MobileNetV3 convolutional network, to obtain the corresponding first facial feature information. In addition, the first device can also use other methods to extract facial features, such as graph neural networks, etc., which are not limited here.

[0060] In some embodiments, the first audio feature information may include a first audio emotion score. The first device can extract audio features based on the first audio data to obtain the first audio features. Then, the first audio features are compared with audio benchmark features to obtain the corresponding first audio emotion score, which can be used to indicate the emotional state of the target customer's speech.

[0061] For example, the first device can extract features from the first audio data through a processing module, and then compare the obtained first audio feature information with the audio soothing benchmark features to obtain a corresponding first soothing score. This first soothing score can be used to indicate the soothing degree of the first audio data, and further, it can indicate the emotional state of the target customer. The more soothing the audio data corresponding to the target customer is, the higher the value of the soothing score; the more urgent the audio data corresponding to the target customer is, the lower the value of the soothing score.

[0062] In one possible implementation, the first device can extract audio features from the first audio data using a Hidden Markov Model to obtain the corresponding first audio feature information. Alternatively, the first device can also use other methods for facial feature extraction, such as Gaussian mixture models or K-nearest neighbor methods, to extract audio features from the first audio data and obtain the corresponding first audio feature information; the specific method is not limited here.

[0063] S203. The first device determines the first state information corresponding to the target customer based on the first facial feature information and the first audio feature information. The first state information is used to indicate the emotional state of the target customer.

[0064] In one possible implementation, the first facial feature information includes a first facial emotion score, the first audio feature information includes a first audio emotion score, the first state information includes a first emotion score, and the first device can determine the first emotion score corresponding to the target customer based on the first facial emotion score, the first weight corresponding to the first facial emotion score, the first audio emotion score, and the second weight corresponding to the first audio emotion score.

[0065] In this possible implementation, machine algorithms replace subjective human judgment to achieve an objective quantitative assessment of customer sentiment, ensuring the timeliness and accuracy of bank service responses.

[0066] It is understandable that both facial emotion scores and audio emotion scores can be used to indicate the emotional state of a target customer. Therefore, when the first device determines the first emotion score corresponding to the target customer based on the first facial emotion score and the first audio emotion score, it needs to combine the first weight corresponding to the first facial emotion score and the second weight corresponding to the first audio emotion score to determine the first emotion score corresponding to the target customer.

[0067] For example, when the first device determines the first emotion score corresponding to the target customer based on the first facial emotion score and the first audio emotion score, the first device can preset corresponding weight values ​​for the facial emotion score and the audio emotion score respectively. The sum of the first weight value corresponding to the facial emotion score and the second weight value corresponding to the audio emotion score is 1. The first device can determine the product of the facial emotion score and the first weight value, and the product of the audio emotion score and the second weight value. Then the first device can determine the sum of the two products as the first emotion score corresponding to the target customer.

[0068] For example, if the first facial emotion score is 60 points, the corresponding first weight is 0.6; if the first audio emotion score is 70 points, the corresponding second weight is 0.4. Then the first emotion score for the target customer is 60*0.6+70*0.4=64.

[0069] In addition, other methods can be used to determine the first emotion score of the target customer based on the first facial emotion score and the first audio emotion score, but specific methods are not limited here.

[0070] In this embodiment of the application, the first emotion score indicating the target customer can be determined by the first facial emotion score and the first audio emotion score. In addition, if other information (such as temperature information) is also included, the first device can also combine other information to determine the first emotion score of the target customer. The specific details are not limited here.

[0071] In this embodiment of the application, the scoring criteria used for different customers can be the same, that is, the audio reference features and facial reference features used are the same, and the scoring method is also the same.

[0072] S204. When the first condition is met, the first device sends first information to the first terminal. The first information is used to instruct the first terminal to interact with the target customer. The first condition includes the first status information satisfying the second condition.

[0073] Accordingly, the first terminal receives the first information from the first device.

[0074] In one possible implementation, the first condition may include first state information satisfying the second condition.

[0075] In this embodiment of the application, when the first condition is met, the first device will send the first information to the first terminal, instructing the first terminal to interact with the target customer. Combined with the first state information, it is used to indicate the emotional state of the target customer. Therefore, the first state information satisfies the second condition, which can be understood as judging based on the emotional state of the target customer, and obtaining that the target customer needs the first terminal to interact with the target customer.

[0076] For example, the first state information may include a first emotion score. If the first state information meets a second condition, it may be that the first emotion score is lower than a preset first threshold. That is, if the first emotion score is lower than the first threshold, the first device sends the first information to the first terminal.

[0077] For example, if the target customer's emotion score is out of 100, and the preset first threshold is 70, and the first emotion score is 64, which is lower than the preset first threshold of 70, the first device sends the first information to the first terminal.

[0078] In addition, the first state information can satisfy the second condition in other ways. For example, the first emotion score of the first state information includes multiple emotion scores of the target customer over a period of time. The variance of these multiple emotion scores is greater than the variance threshold, which can be understood as the target customer's emotions fluctuating greatly. The specifics are not limited here.

[0079] In one possible implementation, the first condition includes receiving a first operation from a first staff member. This first operation instructs the smart terminal to interact with a target customer.

[0080] Understandably, in some scenarios, the first device can interact with the first staff member. The first staff member may deem it necessary for the smart terminal to interact with the target customer, at which point the first staff member can perform a first operation on the first device. Correspondingly, the first device can respond to the first staff member's first operation by sending first information to the first terminal, instructing it to interact with the target customer.

[0081] For example, such as Figure 3 As shown, the first device is equipped with an interactive device at the counter window of the first staff member. For example, it may have a first switch for inputting signals. When the first staff member is serving a target customer and finds that the target customer is interested in other banking services or has many questions about other banking services, the first staff member, after obtaining the customer's consent, can press the first switch to send a first signal to the first device, instructing the first device to assign a smart terminal to interact with the target customer. Then, the first device can send first information through the communication module box of the first smart terminal.

[0082] In addition, staff may also determine in other situations that it is necessary for smart terminals to interact with target customers, but this is not limited here.

[0083] In this embodiment of the application, the first condition may include the first state information satisfying the second condition and / or the receipt of a first operation from the first staff member. In addition, it may include other conditions, which are not limited here.

[0084] In one possible implementation, given that a first condition is met, there may be multiple candidate smart terminals. The first device needs to determine one smart terminal from among these candidate smart terminals and send first information to it, thus determining the first terminal. The first device can determine the first terminal from among the multiple smart terminals based on the distance of each smart terminal from the target customer.

[0085] For example, the first device can first obtain the current location of each of the multiple smart terminals. For instance, the first device can send a location acquisition request to each smart terminal, and then each smart terminal responds to the location acquisition request by sending its own location to the first device. Alternatively, the first device can determine the current location of each smart terminal through other means, such as obtaining the current location of each smart terminal through a camera module; the specific method is not limited here.

[0086] Then, the first device can determine the smart terminal with the smallest distance from the target customer's current location among multiple smart terminals as the first terminal.

[0087] In this embodiment of the application, when the first device needs to determine the first terminal from multiple candidate smart terminals, it can select the smart terminal closest to the target customer's current location as the first terminal, or it can select according to other criteria, such as selecting the smart terminal with the shortest response time as the first terminal. The specific criteria are not limited here.

[0088] In one possible implementation, there might be a situation where at least one smart terminal needs to be assigned to multiple customers. The first device can assign smart terminals to these customers based on their priorities, which can be determined by their emotion scores. Therefore, when assigning a smart terminal to a target customer, the first device can first determine the target customer's priority among the multiple customers based on their first emotion score and the emotion scores of the other customers. Customers with higher emotion scores have higher priority, and customers with lower emotion scores have lower priority. Then, the first device determines the first terminal corresponding to the target customer based on their priority among the multiple customers.

[0089] In this possible implementation, intelligent robots are scheduled based on emotion priority to ensure that high-demand customers receive services first, thereby improving resource utilization.

[0090] In one possible implementation, the first information may include interaction instruction information. The first information is used to instruct the first terminal to interact with the target customer according to the interaction instruction information. That is, the first information can instruct the first terminal to interact with the target customer through the interaction instruction information.

[0091] In one possible implementation, the interaction instruction information can be determined based on the first audio data. Specifically, if a first condition is met, the first device can determine the corresponding interaction instruction information based on the first condition.

[0092] For example, the first device can extract the corresponding text information based on the first voice data, and then compare the text information with the pre-determined business keywords to obtain the keywords in the text information and the frequency of the keywords, thereby determining the corresponding business type.

[0093] For example, if the obtained text information includes frequently occurring keywords such as "interest," "fixed term," and "interest rate," it can be determined that the business the target customer is questioning or needs to inquire about is a fixed-term deposit business; if the obtained text information includes frequently occurring keywords such as "not received," "interbank," and "delay," it can be determined that the business the target customer is questioning or needs to inquire about is an interbank transfer business.

[0094] Then, the first device can determine the set of common business questions corresponding to the sub-categories of the business type, and generate multiple question titles for customers to choose from. For example, for fixed deposit business, the first device can generate titles such as "Current Fixed Deposit Interest Rate Table" and "Interest Rate Promotion Activities".

[0095] It is understood that the interaction instruction information may include at least one of these keywords, business types, and business type-related question titles. In addition, it may include other information that instructs the first terminal to interact with the target customer, which is not limited here.

[0096] In this possible implementation, customer problems can be quickly located by extracting voice keywords and matching business segmentation types, reducing manual intervention and waiting time.

[0097] In one possible implementation, the first information may include first state information, which may include a first emotion score.

[0098] In addition, the first piece of information may include other information, which is not specified here.

[0099] In one possible implementation, the first device can establish a wireless communication connection with the first terminal, such as a Bluetooth connection or a Wi-Fi connection, which is not limited here.

[0100] S205. After receiving the first information, the first terminal can interact with the target customer based on the first information and obtain the first interaction result.

[0101] In one possible implementation, the first information includes interaction instruction information, which indicates the specific content of the interaction between the first terminal and the target customer. For example, the interaction instruction information may include the business type corresponding to the target customer and / or the question title related to the business type. Therefore, the first terminal can interact with the target customer according to the business type corresponding to the target customer and / or the question title related to the business type.

[0102] In one possible implementation, the first terminal includes a walking positioning module, which can move the terminal. After receiving the first information, the first terminal can move to the current location of the target customer, such as the counter location corresponding to the target customer, according to the instructions of the first information.

[0103] In one possible implementation, such as Figure 4 As shown, the first terminal may include an obstacle avoidance module. During the movement of the first terminal, it can avoid obstacles and reach its destination through the obstacle avoidance module.

[0104] In one possible implementation, the first terminal can be mobile. After completing the interaction with the target customer, the first terminal can enter a preset charging area to charge the battery in the first terminal.

[0105] In some embodiments, such as Figure 4 As shown, the first terminal includes a human-computer interaction module, which can be used to interact with target customers.

[0106] For example, the human-computer interaction module may include a display touch screen, a microphone, and a speaker. The first terminal can display preset interactive content to the target customer through the display touch screen, such as the title of a question related to the business that the target customer needs to handle; or play preset interactive audio to the target customer through the speaker.

[0107] Furthermore, the first terminal can execute corresponding response operations based on the target user's actions on the first terminal. For example, if the user touches the integrated touchscreen, the corresponding displayed content is one of the business question titles. Alternatively, the first terminal can acquire the target user's voice information through a microphone. This voice information can be a question about the relevant business or an answer to an audio message corresponding to a business question played by the first terminal through its speaker.

[0108] Correspondingly, in response to the target customer's interactive operations on the first terminal, the first terminal can update the interactive content accordingly. For example, the response content (i.e., the answer to the question) corresponding to the business question title can be displayed on the integrated touch screen. Specifically, this could be a detailed explanation of the business-related regulations or a business-related image file. This resolves the customer's business problem and enables interaction between the first terminal and the customer.

[0109] In some embodiments, the first terminal can perform a query function for relevant business information of the target customer. Specifically, the first terminal can determine the query question indicated by the customer through the interaction module, then generate a corresponding query instruction and send it to the first device. The first device sends the information corresponding to the query instruction (i.e., the information that the customer needs to query) to the first terminal, and then the first terminal displays or plays the information through the interaction module.

[0110] For example, the first terminal can determine the corresponding query question through the customer's touch operation on the integrated touch screen, such as querying a financial product or querying business data of its own bank account. Then, the first terminal sends a query command to the first device through the communication module, and the query command includes the query question.

[0111] Since the first device can communicate with the bank's business inquiry system, after receiving the inquiry instruction, it can communicate with the system to obtain the answer to the inquiry question. The answer is then sent to the first terminal, which displays it to the customer via a touchscreen.

[0112] In addition, the first terminal can also enable customers to query business information through other means, which are not limited here.

[0113] In this possible implementation, the first terminal can interact with customers, reducing the non-core workload of counter staff and allowing them to focus on core business operations, thereby improving overall operational efficiency.

[0114] In one possible implementation, the first information includes first state information, and the first terminal includes a shooting unit, a voice recording unit, and other units for acquiring second customer information of the target customer. The first terminal can collect the second customer information of the target customer when interacting with the target customer, thereby obtaining the second state information of the target customer, and then determine the first interaction result based on the second state information, or the first state information and the second state information.

[0115] For example, the first terminal can interact with the target customer according to the interaction instruction information; then collect second customer information, which may include the target customer's second facial image data and the second customer's second audio data; extract facial features based on the second facial image data to obtain corresponding second facial feature information; extract audio features based on the second audio data to obtain corresponding second audio feature information; determine the target customer's corresponding second state information based on the second facial feature information and the second audio feature information, which is used to indicate the target customer's emotional state; finally, determine the first interaction result based on the second state information, or the first state information and the second state information.

[0116] In this embodiment of the application, the method by which the first terminal determines the second status information of the target customer is the same as the method by which the first device determines the first status information corresponding to the target customer, and will not be described again here.

[0117] In one possible implementation, the first terminal can determine the first interaction result based on the second state information.

[0118] For example, if the second state information includes a second emotion score, the first terminal can compare the second emotion score with a preset range to obtain the corresponding interaction result. For example, if the second emotion score is between 0 and 100, a score of [0, 70) corresponds to an interaction result that the problem is not solved, a score of [70, 90) corresponds to an interaction result that the problem is partially solved, and a score of [90, 100] corresponds to an interaction result that the problem is solved.

[0119] Therefore, if the range of the second emotion score is [0, 70), the corresponding first interaction result is that the problem is not solved; if the range of the second emotion score is [70, 90), the corresponding first interaction result is that the problem is partially solved; and if the range of the second emotion score is [90, 100], the corresponding first interaction result is that the problem is solved.

[0120] In one possible implementation, for example, the second state information includes a second emotion score, and the first state information includes a first emotion score. The first terminal can determine the first interaction result based on the second emotion score and the first emotion score.

[0121] For example, if the ratio of the second emotion score to the first emotion score is less than 0.4, that is, the customer's emotion score has decreased by more than 60%, it can be considered that the customer is more cheerful and the problem has been solved. The corresponding first interaction result is that the problem has been solved.

[0122] If the ratio of the second emotion score to the first emotion score is greater than or equal to 0.4 and less than 0.7, that is, the customer's emotion score has decreased by more than 30% but less than 60%, then the customer's emotion can be considered to have become slightly more pleasant, and some problems have been solved. The corresponding first interaction result is that the problem has been partially solved.

[0123] If the ratio of the second emotion score to the first emotion score is greater than or equal to 0.7, that is, the decrease in the customer's emotion score does not exceed 30%, then it can be considered that the customer's emotion has not become pleasant, the problem has not been solved, and the corresponding first interaction result is that the problem has not been solved.

[0124] In this embodiment of the application, the first interaction result can also be determined by other means, which are not limited here.

[0125] The service method provided in this application collects target customer information through a first device and determines the customer's first state information accordingly. This first state information indicates the customer's emotional state, using the device to replace subjective human judgment, achieving an objective and quantitative assessment of customer emotions, ensuring the timeliness and accuracy of service response, and improving bank satisfaction. On the other hand, it also allows interaction with customers through a first terminal, providing services and reducing the problem of tellers being unable to provide timely service due to busy schedules, further meeting customer needs and improving bank satisfaction.

[0126] In the embodiments of this application, such as Figure 5 As shown, after the first terminal interacts with the target customer and obtains the first interaction result, the first terminal can perform different operations based on different interaction results.

[0127] Therefore, optionally, the method 200 in the embodiments of this application further includes:

[0128] S206. Based on the first interaction result, the first terminal determines the corresponding first prompt mode among multiple prompt modes. The first interaction result can be one of "problem unresolved," "problem partially resolved," or "problem resolved." These multiple prompt modes correspond to different colors, brightness levels, and / or flashing frequencies. Then, the first terminal uses an indicator light to provide a prompt in the first prompt mode.

[0129] For example, the multiple prompt modes may include a green light mode, a yellow light mode, and a red light mode. If the first interaction result is that the problem is not resolved, the prompt light of the first terminal will be red. If the first interaction result is that the problem is partially resolved, the prompt light of the first terminal will be yellow. If the first interaction result is that the problem is resolved, the prompt light of the first terminal will be green.

[0130] In addition, other methods can be used to determine the first prompt mode among multiple prompt modes, but specific methods are not limited here.

[0131] S207. If the first interaction result indicates that the target customer's problem has not been resolved, the first terminal sends a third message to the first device, the third message indicating that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem.

[0132] Correspondingly, the first device receives the third information sent by the first terminal.

[0133] Understandably, if the first terminal fails to resolve the target customer's problem, the first terminal may send third information to the first device so that the first device can know the specific situation.

[0134] In one possible implementation, after receiving the third information sent by the first terminal, the first device takes corresponding actions based on the third information.

[0135] For example, after receiving the third information sent by the first terminal, the first device can confirm that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem. Therefore, the first device can generate a corresponding manual dispatch instruction, which is used to instruct a second staff member to provide service to the target customer and manually answer the target customer's questions.

[0136] For example, the manual dispatch instruction may include relevant business issue information of the target customer, such as business type, business issue title, etc. In addition, it may include other information, such as business segmentation data, counter identification, target customer location, etc., which are not limited here.

[0137] In some embodiments, the first staff member may be a staff member located at the bank counter, and the second staff member may be a staff member located in the lobby.

[0138] In some embodiments, after generating a manual dispatch instruction, the first terminal can send the manual dispatch instruction to a second terminal, which may be a tablet computer or a smartphone.

[0139] Figure 6 This is a schematic diagram of the structure of a first device provided in an embodiment of this application, as shown below. Figure 6 As shown, the first device 600 provided in this embodiment may include a first transceiver module 601 and a first processing module 602.

[0140] For example, the first device 600 includes a first transceiver module 601 and a first processing module 602.

[0141] The first transceiver module 601 is used to collect target customer information, which includes the target customer's first facial image data and the target customer's first audio data.

[0142] The first processing module 602 is used to extract facial features based on the first facial image data to obtain the corresponding first facial feature information.

[0143] The first processing module 602 is also used to extract audio features based on the first audio data to obtain the corresponding first audio feature information.

[0144] The first processing module 602 is further configured to determine the first state information corresponding to the target customer based on the first facial feature information and the first audio feature information, wherein the first state information is used to indicate the emotional state of the target customer.

[0145] The first transceiver module 601 is further configured to send first information to the first terminal when a first condition is met. The first information is used to instruct the first terminal to interact with the target customer. The first condition includes the first status information satisfying a second condition.

[0146] In one possible implementation, the first facial feature information includes a first facial emotion score, and the first processing module 602 is specifically used to extract facial features based on the first facial image data to obtain the corresponding first facial emotion score; the first audio feature information includes a first audio emotion score, and the first processing module 602 is specifically used to extract audio features based on the first audio data to obtain the corresponding first audio emotion score; the first state information includes a first emotion score, and the first processing module 602 is specifically used to determine the first emotion score corresponding to the target customer based on the first facial emotion score, the first weight corresponding to the first facial emotion score, the first audio emotion score, and the second weight corresponding to the first audio emotion score.

[0147] In one possible implementation, the first processing module 602 is further configured to determine interaction instruction information based on the first audio data. The interaction instruction information includes the business type corresponding to the target customer and / or the question title related to the business type. The first information includes the interaction instruction information, which is used to instruct the first terminal to interact with the target customer based on the interaction instruction information.

[0148] In one possible implementation, the first information also includes first state information.

[0149] In one possible implementation, the first transceiver module 601 is further configured to obtain the current location of each of the multiple smart terminals; the first processing module 602 is further configured to determine a first terminal among the multiple smart terminals based on the current location of each smart terminal and / or a first emotion score, wherein the first terminal is the smart terminal among the multiple smart terminals that is the one with the smallest distance from the current location of the target customer.

[0150] In one possible implementation, the first processing module 602 is further configured to determine the priority of the target customer among multiple customers based on the first emotion score; the first processing module 602 is further configured to determine the first terminal corresponding to the target customer based on the priority of the target customer among multiple customers.

[0151] In one possible implementation, the first condition further includes: receiving a first operation from a first staff member, the first operation being used to instruct the smart terminal to interact with the target customer.

[0152] In one possible implementation, the first transceiver module 601 is further configured to receive third information from the first terminal, the third information indicating that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem.

[0153] Figure 7 This is a schematic diagram of the structure of a first terminal provided in an embodiment of this application, such as... Figure 7 As shown, the first terminal 700 provided in this embodiment includes: a second transceiver module 701 and a second processing module 702.

[0154] The second transceiver module 701 is used to receive first information, which instructs the first terminal to interact with the target customer.

[0155] The second processing module 702 is used to interact with the target customer based on the first information and obtain the first interaction result.

[0156] In one possible implementation, the first information includes interaction instruction information, which includes the business type corresponding to the target customer and / or the question title related to the business type. The second processing module 702 is specifically used to: interact with the target customer based on the business type corresponding to the target customer and / or the question title related to the business type.

[0157] In one possible implementation, the first information further includes first state information. The second processing module 702 is specifically used to: interact with the target customer according to the interaction instruction information; the second transceiver module 701 is specifically used to collect second customer information, which includes second facial image data and second audio data of the target customer; the second processing module 702 is specifically used to: extract facial features based on the second facial image data to obtain corresponding second facial feature information; the second processing module 702 is specifically used to: extract audio features based on the second audio data to obtain corresponding second audio feature information; the second processing module 702 is specifically used to: determine the second state information corresponding to the target customer based on the second facial feature information and the second audio feature information, the second state information indicating the target customer's emotional state; and the second processing module 702 is specifically used to: determine the first interaction result based on the second state information, or the first state information and the second state information.

[0158] In one possible implementation, the second transceiver module 701 is further configured to: send third information to the first device if the first interaction result indicates that the target customer's problem has not been resolved, the third information indicating that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem.

[0159] In one possible implementation, the first terminal includes an indicator light, and the second processing module 702 is further configured to determine a corresponding first prompt mode among multiple prompt modes based on the first interaction result. The first interaction result is one of the following: the problem is not resolved, the problem is partially resolved, and the problem is resolved. The multiple prompt modes correspond to different colors, different brightness, and / or different flashing frequencies.

[0160] The second processing module 702 is also used to provide prompts via indicator lights in the first prompt mode.

[0161] The first device and the first terminal provided in this embodiment can execute the methods provided in the above method embodiments. Their implementation principles and technical effects are similar, and will not be described in detail here.

[0162] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 80 may include a memory 801 and a processor 802. Optionally, the electronic device may also include a transceiver 803, wherein the memory 801 and the processor 802 communicate; for example, the memory 801, the processor 802, and the transceiver 803 may communicate via a communication bus 804, the memory 801 is used to store a computer program, and the processor 802 executes the computer program to implement the method performed by the first device, the first terminal, or the second terminal in the above embodiments.

[0163] Optionally, the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps in the method embodiments disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0164] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods in any of the above method embodiments.

[0165] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above method embodiments.

[0166] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0167] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0168] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0170] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0171] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0172] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0173] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0174] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0175] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0176] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0177] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0178] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0179] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0180] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A service method, characterized in that, The method includes: Collect target customer information, which includes the target customer's first facial image data and the target customer's first audio data; Based on the first facial image data, facial features are extracted to obtain the corresponding first facial feature information; Based on the first audio data, audio features are extracted to obtain the corresponding first audio feature information; The first state information corresponding to the target customer is determined based on the first facial feature information and the first audio feature information, and the first state information is used to indicate the emotional state of the target customer. If a first condition is met, first information is sent to a first terminal, the first information being used to instruct the first terminal to interact with the target customer, the first condition including the first status information satisfying a second condition.

2. The method according to claim 1, characterized in that, The first facial feature information includes a first facial emotion score. The step of extracting facial features based on the first facial image data to obtain the corresponding first facial feature information includes: Based on the first facial image data, facial features are extracted to obtain the corresponding first facial emotion score. The first audio feature information includes a first audio emotion score. The step of extracting audio features based on the first audio data to obtain the corresponding first audio feature information includes: Based on the first audio data, audio features are extracted to obtain the corresponding first audio emotion score; The first state information includes a first emotion score, and the step of determining the first state information corresponding to the target customer based on the first facial feature information and the first audio feature information includes: The first emotion score corresponding to the target customer is determined based on the first facial emotion score, the first weight corresponding to the first facial emotion score, the first audio emotion score, and the second weight corresponding to the first audio emotion score.

3. The method according to claim 2, characterized in that, The method further includes: Based on the first audio data, interactive instruction information is determined, which includes the business type corresponding to the target customer and / or the question title related to the business type. The first information includes the interaction instruction information, which is used to instruct the first terminal to interact with the target customer according to the interaction instruction information.

4. The method according to claim 3, characterized in that, The first information also includes the first status information.

5. The method according to claim 4, characterized in that, Before sending the first information to the first terminal, the method further includes: Get the current location of each of the multiple smart terminals; Based on the current location of each smart terminal and / or the first emotion score, the first terminal is determined among a plurality of smart terminals, wherein the first terminal is the smart terminal with the smallest distance from the current location of the target customer among the plurality of smart terminals.

6. The method according to claim 4, characterized in that, Before sending the first information to the first terminal, the method further includes: The priority of the target customer among multiple customers is determined based on the first emotion score; The first terminal corresponding to the target customer is determined based on the priority of the target customer among multiple customers.

7. The method according to any one of claims 1-6, characterized in that, The first condition also includes: The system receives a first operation from a first staff member, the first operation being used to instruct the smart terminal to interact with the target customer.

8. The method according to claim 7, characterized in that, The method further includes: Receive third information from the first terminal, the third information indicating that the interaction between the first terminal and the target customer failed to resolve the target customer's problem.

9. A service method, characterized in that, The method includes: Receive first information, the first information instructing the first terminal to interact with the target customer; Based on the first information, interact with the target customer to obtain a first interaction result.

10. The method according to claim 9, characterized in that, The first information includes interaction instruction information, which includes the business type corresponding to the target customer and / or the question title related to the business type. The step of interacting with the target customer based on the first information to obtain a first interaction result includes: Interact with the target customer based on the business type corresponding to the target customer and / or the question titles related to the business type.

11. The method according to claim 9 or 10, characterized in that, The first information also includes first status information, and the step of interacting with the target customer according to the interaction instruction information to obtain a first interaction result includes: Interact with the target customer according to the interaction instruction information; Collect second customer information, which includes second facial image data of the target customer and second audio data of the second customer; Based on the second facial image data, facial features are extracted to obtain the corresponding second facial feature information; Based on the second audio data, audio features are extracted to obtain the corresponding second audio feature information; The second state information corresponding to the target customer is determined based on the second facial feature information and the second audio feature information, and the second state information is used to indicate the emotional state of the target customer. The first interaction result is determined based on the second status information, or the first status information and the second status information.

12. The method according to claim 11, characterized in that, The method further includes: If the first interaction result indicates that the target customer's problem has not been resolved, a third message is sent to the first device, the third message indicating that the interaction between the first terminal and the target customer has failed to resolve the target customer's problem.

13. The method according to claim 11, characterized in that, The first terminal includes an indicator light, and the method further includes: Based on the first interaction result, a corresponding first prompt mode is determined among multiple prompt modes. The first interaction result is one of the following: problem unresolved, problem partially resolved, and problem resolved. The multiple prompt modes correspond to different colors, different brightness, and / or different flashing frequencies. The indicator light provides a notification in the first notification mode.

14. A service device, characterized in that, include: The first transceiver module and the first processing module; The first transceiver module is used to collect target customer information, which includes the target customer's first facial image data and the target customer's first audio data. The first processing module is used to extract facial features based on the first facial image data to obtain corresponding first facial feature information; The first processing module is further configured to extract audio features based on the first audio data to obtain corresponding first audio feature information; The first processing module is further configured to determine the first state information corresponding to the target customer based on the first facial feature information and the first audio feature information, wherein the first state information is used to indicate the emotional state of the target customer; The first transceiver module is further configured to send first information to the first terminal when a first condition is met. The first information is used to instruct the first terminal to interact with the target customer. The first condition includes the first status information satisfying a second condition.

15. A service device, characterized in that, include: Second transceiver module and second processing module; The second transceiver module is used to receive first information, which instructs the first terminal to interact with the target customer; The second processing module is used to interact with the target customer based on the first information to obtain a first interaction result.

16. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as claimed in any one of claims 1 to 8 or the method as claimed in any one of claims 9 to 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8 or the method as described in any one of claims 9 to 13.

18. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8 or the method of any one of claims 9 to 13.

19. A service system, characterized in that, It includes a first device and a first terminal, the first device being used to perform the method as described in any one of claims 1 to 8, and the first terminal being used to perform the method as described in any one of claims 9 to 13.

Images