Data processing method and device and customer service robot

Abstract

The specification provides a data processing method and device and a customer service robot, and applies to the technical field of artificial intelligence. The customer service robot monitors ambient sound, and in a case where it is determined that a preset triggering condition is met, acquires and determines a current service type according to first target data of a target customer; determines a matching target processing rule from preset processing rules according to the current service type; acquires and determines whether the target customer has a service requirement according to second target data about the target customer according to the target processing rule; and in a case where it is determined that the target customer has a service requirement, interacts with the target customer according to a preset interaction rule to provide matching service for the target customer. Thus, different service scenarios can be effectively and comprehensively covered, customers with service requirements can be accurately detected and identified, and interaction is actively performed, so that customers can obtain a better service experience.

Description

Technical Field

[0001] This manual belongs to the field of artificial intelligence technology, and in particular relates to data processing methods, devices and customer service robots. Background Technology

[0002] In the field of artificial intelligence, with the development and popularization of robotics, more and more robots are being used in customer service applications. For example, customer service robots are often deployed in the business halls of banks and other institutions to answer customer inquiries or assist customers with related business.

[0003] However, based on existing methods, customer service robots can only be activated to serve customers when they speak keywords or are touched by them. Based on existing methods, customer service robots cannot proactively and accurately identify customers with service needs and actively interact with them, thus impacting the customer service experience.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This manual provides a data processing method, device, and customer service robot that can effectively and comprehensively cover different business service scenarios, accurately detect and identify customers with business service needs, and proactively interact with customers without requiring them to wake up, so as to intelligently provide relevant business services to customers and enable them to obtain a better service experience.

[0006] This manual provides a data processing method for use in customer service robots, including:

[0007] Monitor ambient sound to determine if the preset triggering conditions are met;

[0008] If the preset triggering conditions are met, obtain the first target data about the target customer;

[0009] Based on the first target data, determine the current business service type;

[0010] Based on the current business service type, determine the matching target processing rule from the preset processing rules;

[0011] Based on the target processing rules, obtain secondary target data about the target customer;

[0012] Based on the target processing rules, the second target data is processed to determine whether the target customer has a business service need;

[0013] Once it is determined that the target user has a need for business services, the system interacts with the target customer according to preset interaction rules in order to provide the target customer with matching business services.

[0014] In one embodiment, monitoring ambient sound to determine whether a preset trigger condition is met includes:

[0015] Collect the current ambient sound and perform human voice detection on the current ambient sound to determine whether there is a human voice in the current ambient sound;

[0016] If it is determined that there is human voice in the current ambient sound, extract the human voice audio from the current ambient sound.

[0017] Detect whether the human voice audio is interference sound;

[0018] If it is determined that the human voice audio is not interference, it is determined that a target customer exists in the current environment, and the preset triggering conditions are met.

[0019] In one embodiment, the interference sound includes at least one of the following: broadcast sound, staff voice, or meaningless human voice.

[0020] In one embodiment, determining the current business service type based on the first target data includes:

[0021] The first audio data in the first target data is processed using a preset speech recognition model to obtain the corresponding first text;

[0022] Detect whether the first text contains preset keywords or related words.

[0023] If it is determined that there are preset keywords or related words in the first text, the current business service type is determined to be the first type of business service; wherein, the first type of business service is the business service type in which customers inquire about business issues and wait for a reply.

[0024] In one embodiment, obtaining second target data about the target customer according to the target processing rules includes:

[0025] According to the target processing rules, audio data within the first time period after the first audio data is collected is used as the second audio data in the second target data.

[0026] And / or,

[0027] According to the target processing rules, the current location of the target customer is determined using the first audio data; according to the preset time interval, multiple images are taken at the current location of the target customer, which are used as the second image data in the second target data.

[0028] In one embodiment, determining whether a target customer has a business service need by processing second target data according to target processing rules includes:

[0029] The second audio data in the second target data is processed using a preset speech recognition model to obtain the corresponding second text;

[0030] Detect whether there is a response text in the second text that is related to preset keywords or related terms;

[0031] If the response text is not found in the second text, it is determined that the target customer has a business service need.

[0032] In one embodiment, determining whether a target customer has a business service need by processing second target data according to target processing rules includes:

[0033] Based on the second image data in the second target data, detect whether there are staff members at the current location of the target customer;

[0034] If no staff are present at the target customer's current location, it is determined that the target customer has a need for business services.

[0035] In one embodiment, determining the current business service type based on the first target data further includes:

[0036] Based on the first image data in the first target data, detect whether the target customer is currently using a self-service machine to conduct self-service business;

[0037] If it is determined that the target customer is currently using a self-service machine to conduct self-service business, the current business service type is determined to be the second type of business service; wherein, the second type of business service is the business service type in which the customer uses a self-service machine to conduct self-service business.

[0038] In one embodiment, obtaining second target data about the target customer according to the target processing rules includes:

[0039] According to the target processing rules, video data within a second time period following the first image data is recorded in the direction of the self-service machine, and this data is used as the second image data in the second target data.

[0040] In one embodiment, determining whether a target customer has a business service need by processing second target data according to target processing rules includes:

[0041] Based on the second image data in the second target data, detect whether the target customer encounters any problems when using the self-service machine to handle self-service business;

[0042] If it is determined that the target customer is experiencing problems while using the self-service machine to conduct self-service business, then it is determined that the target customer has a business service need.

[0043] In one embodiment, interacting with the target customer according to preset interaction rules includes:

[0044] Generate matching interactive text based on preset interaction rules;

[0045] Generate corresponding interactive audio based on the interactive text;

[0046] Play the interactive audio to the target customer.

[0047] In one embodiment, while playing the interactive audio to the target customer, the method further includes:

[0048] According to preset interaction rules, the indicator light flashes at a matching frequency to alert the target customer.

[0049] This specification also provides a data processing device for use in customer service robots, including:

[0050] The monitoring module is used to monitor ambient sound to determine whether the preset triggering conditions are met.

[0051] The first acquisition module is used to acquire first target data about the target customer when it is determined that the preset triggering conditions are met.

[0052] The first determining module is used to determine the current business service type based on the first target data;

[0053] The second determination module is used to determine the matching target processing rule from the preset processing rules based on the current business service type.

[0054] The second acquisition module is used to acquire second target data about the target customer according to the target processing rules;

[0055] The third determination module is used to determine whether the target customer has a business service need by processing the second target data according to the target processing rules.

[0056] The interaction module is used to interact with the target customer according to preset interaction rules when it is determined that the target user has a business service need, so as to provide the target customer with matching business services.

[0057] This specification also provides a customer service robot, including a processor and a memory for storing processor-executable instructions, wherein the processor executes the instructions to implement the relevant steps of the data processing method.

[0058] This specification also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the relevant steps of the data processing method.

[0059] Based on the data processing method, apparatus, and customer service robot provided in this manual, the customer service robot monitors ambient sound. When a preset triggering condition is met, it acquires and, based on first target data about the target customer, uses multi-dimensional data through multimodal fusion to determine the current business service type. Then, based on the current business service type, it determines a matching target processing rule from preset processing rules. According to the target processing rule, it acquires and, based on second target data about the target customer, uses multi-dimensional data through multimodal fusion to determine if the target customer has a business service need. If the target customer has a business service need, it interacts with the target customer according to preset interaction rules to provide matching business services. By first differentiating different business service types based on the first target data, and then acquiring and utilizing the second target data according to the matching target processing rules for different business server types, it accurately determines whether the target customer truly has a business service need. This allows for more effective and comprehensive coverage of different business service scenarios, accurately detecting and identifying customers with business service needs, and proactively interacting with customers without requiring them to be activated, intelligently providing relevant business services and ensuring a better customer experience. Attached Figure Description

[0060] To more clearly illustrate the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a data processing method provided in one embodiment of this specification;

[0062] Figure 2 This is a schematic diagram illustrating one embodiment of the data processing method provided in the embodiments of this specification, applied in a scenario example.

[0063] Figure 3 This is a schematic diagram illustrating one embodiment of the data processing method provided in the embodiments of this specification, applied in a scenario example.

[0064] Figure 4 This is a schematic diagram illustrating one embodiment of the data processing method provided in the embodiments of this specification, applied in a scenario example.

[0065] Figure 5 This is a schematic diagram illustrating one embodiment of the data processing method provided in the embodiments of this specification, applied in a scenario example.

[0066] Figure 6 This is a schematic diagram of the structural composition of a customer service robot provided in one embodiment of this specification;

[0067] Figure 7 This is a schematic diagram of the structural composition of a data processing apparatus provided in one embodiment of this specification. Detailed Implementation

[0068] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0069] See Figure 1 As shown in the embodiments of this specification, a data processing method is provided, which is specifically applied to the customer service robot side. In specific implementation, the method may include the following:

[0070] S101: Monitor ambient sound to determine whether the preset trigger conditions are met;

[0071] S102: If the preset triggering conditions are met, obtain the first target data about the target customer;

[0072] S103: Determine the current business service type based on the first target data;

[0073] S104: Based on the current business service type, determine the matching target processing rule from the preset processing rules;

[0074] S105: Obtain secondary target data about the target customer according to the target processing rules;

[0075] S106: Based on the target processing rules, determine whether the target customer has business service needs by processing the second target data;

[0076] S107: When it is determined that the target user has a business service need, interact with the target customer according to the preset interaction rules to provide the target customer with matching business services.

[0077] Based on the above embodiments, when the preset triggering conditions are met, the current business service type is first obtained and determined according to the first target data; then, different business service types are distinguished, and for the current different business service types, based on the matching target processing rules, the second target data about the target customer is obtained and accurately determined whether the target customer really has a business service need; thus, when it is determined that the target customer has a business service need, the system can proactively interact with the customer without requiring the customer to wake up, and intelligently provide the customer with relevant business services, thereby enabling the customer to obtain a better service experience.

[0078] In some embodiments, the data processing method described above can be specifically applied to a customer service robot. Specifically, the customer service robot may include an intelligent robot deployed in the business halls of institutions such as banks, used to provide customers with diverse business services related to the bank or other institutions.

[0079] Of course, it should be noted that the bank and other institution service halls listed above are only illustrative examples. In practice, depending on the specific circumstances and business needs, these customer service robots can be deployed and applied to other suitable locations. For example, they can be used in hotel lobbies or shopping malls to provide customers with more diversified business services.

[0080] In some embodiments, see Figure 2 As shown, the aforementioned customer service robot includes at least a processor, a display screen, a microphone, a camera, and a voice player. Specifically, in order to better monitor ambient sound and collect audio data, the microphone can be a microphone array.

[0081] Furthermore, to handle more complex customer service interaction scenarios, the aforementioned customer service robot can also be configured with a signal transceiver. (See also...) Figure 3 As shown, the aforementioned customer service robot can also interact with the cloud server via a signal transceiver. In this way, the customer service robot can focus solely on collecting ambient sound, audio data, and video data, and then transmit this data to the cloud server. The server processes the data, obtains the corresponding processing results, and then transmits these results back to the customer service robot. The customer service robot can then interact with the customer based on the received processing results.

[0082] Specifically, the cloud server may include a backend server applied to one side of a cloud computing platform, capable of data transmission, data processing, and other functions. Specifically, the cloud server may be, for example, an electronic device with data computing, storage, and network interaction capabilities. Alternatively, the cloud server may be a software program running on the electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers included in the cloud server is not specifically limited. The cloud server may be a single server, several servers, or a server cluster formed by several servers.

[0083] In some embodiments, see Figure 4 As shown, the above-mentioned monitoring of ambient sound to determine whether the preset triggering conditions are met can, in specific implementation, include the following:

[0084] S1: Collect the current ambient sound and perform human voice detection on the current ambient sound to determine whether there is a human voice in the current ambient sound;

[0085] S2: If it is determined that there is human voice in the current ambient sound, extract the human voice audio from the current ambient sound;

[0086] S3: Detect whether the human voice audio is interference sound;

[0087] S4: If it is determined that the human voice audio is not interference sound, it is determined that there is a target customer in the current environment and that the preset triggering conditions are met.

[0088] Specifically, the aforementioned target customers can be understood as customers who may have business service needs but require further confirmation. For these target customers, the customer service robot can further track and follow up to confirm whether they genuinely have a business service need.

[0089] Based on the above embodiments, the customer service robot can set its microphone to always be on, monitor ambient sounds, detect whether there are potential customers in the current environment who may have business service needs, and determine whether the preset triggering conditions are met.

[0090] In some embodiments, VAD (Voice Activity Detection) technology can be used to monitor the presence of human voices in the ambient sound. If a human voice is detected in the current ambient sound, audio data containing the human voice can be extracted as human voice audio.

[0091] In some embodiments, the aforementioned interference sounds may specifically include at least one of the following: broadcast sounds, staff voices, meaningless human voices, etc.

[0092] Taking the service halls of banks and other institutions as an example, in addition to the customer's voice, there are often other noises in such environments, such as announcements calling numbers, staff members talking, and meaningless interjections used by customers or staff during interactions. Although these are also human voices, customer service robots cannot determine whether a customer has a service request based on these human voices alone. On the contrary, these human voices can interfere with the robot's judgment.

[0093] Based on the above embodiments, by considering and finely distinguishing the interference sounds in the environment, the judgment error of the customer service robot can be effectively reduced, and the preset triggering conditions can be determined more accurately.

[0094] In some embodiments, the detection of whether the human voice audio is interference sound may specifically include the following:

[0095] S1: Match the human voice audio according to the preset sound sample library to obtain the corresponding matching result;

[0096] S2: Determine whether the human voice is interference sound based on the matching result.

[0097] Specifically, the preset sound sample library can store voiceprint samples of staff, common meaningless human voice audio samples, and common broadcast sound samples, etc.

[0098] Before implementation, voiceprint data of staff can be collected to construct voiceprint samples corresponding to each staff member and store them in a preset sound sample library; a large number of historical environmental sound records can be collected, and high-frequency meaningless human voice audio can be selected from the large number of historical environmental sound records as common meaningless human voice audio samples and stored in a preset sound sample library; high-frequency broadcast audio can also be selected from the large number of historical environmental sound records as broadcast audio samples of the Yangtze River and stored in a preset sound sample library.

[0099] In practice, sound features can be extracted from human voice audio; then, these sound features are matched against samples stored in a preset sound sample library. If a match is successful, the extracted human voice audio can be determined to be interference, thus indicating that the preset triggering conditions are not met. Conversely, if a match fails, the extracted human voice audio can be determined to be the customer's voice, thus indicating that the preset triggering conditions are met.

[0100] In some embodiments, the detection of whether the human voice audio is interference may further include the following: processing the human voice audio using a preset interference sound recognition model to determine whether the human voice audio is interference sound.

[0101] Specifically, the aforementioned preset interference sound recognition model can be understood as an algorithm model that can identify whether the input audio is interference sound based on the input audio.

[0102] Before implementation, a large number of interference audio recordings (e.g., staff voice audio, broadcast audio, meaningless human voice audio, etc.) can be collected as positive samples, and a large number of customer voice audio recordings can be collected as negative samples. The positive and negative samples are combined to obtain a training set. The training set is used to train the model to obtain a preset interference sound recognition model.

[0103] In some embodiments, taking the business hall of a bank or other institution as an example, and combining specific business service scenarios, and through statistical analysis of a large number of customer questionnaire survey results, the types of business services that the customer service machine needs to cover can be divided into three categories: the first type of business service, the second type of business service, and the third type of business service.

[0104] The first type of business service specifically refers to the type of business service where customers inquire about business issues and await a response. For this type of business service, if a customer raises a business question but, for some reason, does not receive the desired response within a relatively long period of time, it can be considered that the customer has a business service need. Accordingly, the customer service robot can proactively interact with the customer, answer the relevant business questions, or guide the customer to obtain the required response.

[0105] The second type of service specifically refers to services where customers use self-service kiosks to conduct self-service transactions. For this type of service, if a customer encounters difficulties or problems while using a self-service kiosk and is unable to resolve these difficulties or problems for an extended period, preventing them from successfully completing the self-service transaction, then the customer is considered to have a service need. Accordingly, the customer service robot can proactively interact with the customer to assist or guide them in successfully using the self-service kiosk to conduct their self-service transactions.

[0106] The third type of service specifically refers to services where customers proactively seek assistance from the customer service robot. For this type of service, when a customer shows a tendency to proactively seek help from the robot but hasn't yet given explicit instructions, the robot can proactively interact with the customer before they give clear instructions, thereby improving the customer's service experience.

[0107] In some embodiments, before implementation, a large number of historical service records can be collected; then, based on the historical service records, different business service types can be distinguished to obtain historical service records for each business service type; and the historical service records for each business service type can be learned separately to construct preset processing rules corresponding to each business service type.

[0108] The preset processing rules can include, at a minimum, the methods for acquiring audio and / or video data, as well as limitations on the processing methods. Specifically, the aforementioned impact data can include image data and / or video data.

[0109] In some embodiments, when a preset triggering condition is met, the location information of the target customer can be determined first by using ambient sound based on sound localization technology; then, based on the location information, first target data about the target customer can be collected and obtained.

[0110] In some embodiments, the aforementioned first target data may specifically include first audio data and / or first video data about the target customer. In specific implementations, depending on the specific circumstances and processing needs, the first audio data may be collected alone as the first target data, the first video data may be collected alone as the first target data, or both the first audio data and the first video data may be collected simultaneously as the first target data.

[0111] In some embodiments, determining the current business service type based on the first target data may include the following:

[0112] S1: Process the first audio data in the first target data using a preset speech recognition model to obtain the corresponding first text;

[0113] S2: Detect whether the first text contains preset keywords or related words.

[0114] S3: If it is determined that there are preset keywords or related words in the first text, the current business service type is determined to be the first type of business service; wherein, the first type of business service is the business service type in which customers inquire about business issues and wait for a reply.

[0115] Based on the above embodiments, it is possible to accurately determine whether the current business service type is the first type of business service type by multimodal fusion based on the first target data.

[0116] In some embodiments, the aforementioned preset speech recognition model can be understood as an algorithm model built based on speech recognition technology that can recognize input audio data and convert it into corresponding text data.

[0117] The aforementioned preset keywords can be understood as keywords related to business services.

[0118] Before implementation, keywords related to frequently asked customer questions can be selected as the preset keywords based on the specific business scenario. Related terms can include phrases that are semantically similar to or closely related to the preset keywords. Specifically, after determining the preset keywords, natural language models and other technologies can be used to semantically expand and connect the keywords to identify related terms. Combining the preset keywords and related terms yields a keyword list. The customer service robot can then hold this keyword list.

[0119] Accordingly, in practice, the customer service robot can search the first text based on the keyword list to determine whether the first text contains preset keywords or related words.

[0120] In some embodiments, the aforementioned second target data may specifically include second audio data and / or second video data about the target customer. In specific implementations, depending on the specific circumstances and processing needs, the second audio data may be collected separately as the second target data, the second video data may be collected separately as the second target data, or the second audio data and the second video data may be collected simultaneously as the second target data.

[0121] In some embodiments, the above-mentioned acquisition of second target data about the target customer according to the target processing rules may include the following: according to the target processing rules, collecting audio data within a first time period after the first audio data as the second audio data in the second target data; and / or, according to the target processing rules, determining the current location of the target customer using the first audio data; and taking multiple images toward the current location of the target customer according to a preset time interval as the second image data in the second target data.

[0122] Based on the above embodiments, second target data that is relatively effective for the first type of business service can be collected and obtained in a targeted manner, so as to more accurately determine whether the target customers under the first type of business service have business service needs.

[0123] Specifically, the first time period mentioned above can be 2 minutes, and the preset time interval mentioned above can be 20 seconds. Of course, the first time period and the preset time interval listed above are only illustrative.

[0124] In some embodiments, the above-described determination of whether a target customer has a business service need by processing the second target data according to the target processing rules may include the following:

[0125] S1: Process the second audio data in the second target data using a preset speech recognition model to obtain the corresponding second text;

[0126] S2: Detect whether there is a response text in the second text that is related to the preset keywords or related terms;

[0127] S3: If it is determined that the response text does not exist in the second text, it is determined that the target customer has a business service need.

[0128] Based on the above embodiments, it is possible to obtain and, based on the second audio data, accurately and efficiently determine whether the target customer under the first type of business service truly has a need for the business service.

[0129] In practice, if it is determined that there is no reply text in the second text, it can be concluded that the target customer has not received a response for a considerable period of time after raising a business question. In this case, to avoid affecting the target customer's service experience, the target customer can be identified as a customer with business service needs, and the customer service robot can be automatically activated and proactively interact with the target customer.

[0130] Conversely, if a response is found in the second text, it can be determined that a staff member has already answered the business question raised by the target customer. Accordingly, it can be determined that the target customer has no business needs and there is no need to proactively interact with them. Furthermore, the customer service chatbot can continue monitoring.

[0131] In some embodiments, the above-described determination of whether a target customer has a business service need by processing the second target data according to the target processing rules may include the following:

[0132] S1: Based on the second image data in the second target data, detect whether there are staff members at the current location of the target customer;

[0133] S2: If it is determined that there are no staff members at the target customer's current location, then the target customer has a business service need.

[0134] Based on the above embodiments, it is possible to acquire and, based on the second image data, accurately and efficiently determine whether the target customer under the first type of business service truly has a need for the business service.

[0135] In practice, if the second image data determines that there are no staff members at the target customer's current location, it can be concluded that no staff member has come to serve the target customer within a relatively long initial period after the target customer raised their business question. In this case, to avoid impacting the target customer's service experience, the target customer can be identified as someone with a business service need, and the customer service robot can be automatically activated and proactively interact with the target customer.

[0136] Conversely, if the second image data determines that staff are present at the target customer's current location, it can be concluded that staff are already serving the target customer and answering their business inquiries. Accordingly, it can be determined that the target customer has no business needs and does not require proactive interaction. Furthermore, the customer service robot can continue monitoring.

[0137] In some embodiments, the above-described processing method for determining whether a target customer has a business service need based on the second audio data can be combined with the above-described processing method for determining whether a target customer has a business service need based on the second image data; by using the above two methods to verify each other, multimodal fusion can be performed so as to more accurately determine whether the target customer really has a business service need.

[0138] In some embodiments, the detection of whether there are staff members at the current location of the target customer based on the second image data in the second target data may include the following:

[0139] S1: Perform image recognition on multiple images contained in the second image data to obtain multiple image recognition results;

[0140] S2: Based on the recognition results of each image, detect whether there are any human figures wearing uniforms with distinctive features in each image;

[0141] S3: If it is determined that at least one of the multiple images contains a person wearing a uniform with distinctive features, then the presence of staff is confirmed.

[0142] Specifically, considering that staff are mostly required to wear uniforms, such as black suits, representative identifying features can be determined beforehand based on staff payments. This allows for quick identification of staff presence in an image simply by detecting the presence of such identifying features.

[0143] In some embodiments, when it is determined that there are staff members at the target customer's current location, the method may further include the following:

[0144] S1: Select the target image containing staff from multiple images;

[0145] S2: Perform motion recognition on the staff in the target image to obtain the motion recognition results;

[0146] S3: Based on the action recognition results, if the staff's behavior is determined to be a service behavior, it is determined that the target customer has no business service needs.

[0147] Specifically, keypoint detection can be performed on the human body in the target image to obtain the corresponding keypoint detection results. Then, based on the keypoint detection results, they are matched with the keypoints of the template's behavioral actions to obtain the corresponding action recognition results. In this way, the action recognition results of the staff can be accurately obtained.

[0148] In some embodiments, the determination of the current business service type based on the first target data may further include the following:

[0149] S1: Based on the first image data in the first target data, detect whether the target customer is currently using a self-service machine to conduct self-service business;

[0150] S2: If it is determined that the target customer is currently using a self-service machine to conduct self-service business, the current business service type is determined to be the second type of business service; wherein, the second type of business service is the business service type in which the customer uses a self-service machine to conduct self-service business.

[0151] Based on the above embodiments, it is possible to obtain and accurately determine whether the current business service type is the second type of business service based on the first target data.

[0152] In some embodiments, the above-mentioned acquisition of second target data about the target customer according to the target processing rules may specifically include: recording video data within a second time period after the first image data in the direction of the self-service machine location, according to the target processing rules, as the second image data in the second target data.

[0153] Based on the above embodiments, second target data that is relatively effective for the second type of business service can be collected and obtained in a targeted manner, so as to more accurately determine whether the target customers under the second type of business service have business service needs.

[0154] The second time period can be ten minutes. Of course, in practice, other durations can be set as the second time period depending on the specific circumstances and processing needs.

[0155] In some embodiments, the above-described determination of whether a target customer has a business service need by processing the second target data according to the target processing rules may include the following:

[0156] S1: Based on the second image data in the second target data, detect whether the target customer encounters any problems when using the self-service machine to handle self-service business;

[0157] S2: If it is determined that the target customer has encountered problems when using the self-service machine to handle self-service business, it is determined that the target customer has a business service need.

[0158] Based on the above embodiments, it is possible to acquire and, based on the second image data, accurately and efficiently determine whether the target customer under the second type of business service truly has a need for the business service.

[0159] In some embodiments, the above-mentioned detection of whether a target customer encounters problems while using the self-service machine to conduct self-service business based on the second image data in the second target data may specifically include: analyzing the target customer's facial expression changes within a second time period based on the second image data to determine the target customer's emotional changes; and determining whether the target customer encounters problems while using the self-service machine to conduct self-service business based on the target customer's emotional changes. When it is determined that the target customer is encountering problems while using the self-service machine to conduct self-service business, it can be determined that the target customer has a need for business services.

[0160] In addition, the above-mentioned detection of whether the target customer encounters problems when using the self-service machine to handle self-service business based on the second image data in the second target data may, in specific implementation, include: detecting whether the target customer's stay time at the self-service machine exceeds the reference time based on the second image data; if it is determined that the stay time exceeds the reference time, it can be determined that the target customer has a business service need.

[0161] In some embodiments, the determination of the current business service type based on the first target data may further include the following:

[0162] S1: Based on the initial target data, detect whether the target customer shows a tendency to seek help from the customer service robot; (e.g., the target customer's gaze is focused on the robot, or the target customer moves closer to the customer service robot).

[0163] S2: If a trend of target customers seeking help from customer service robots is detected, the current business service type is determined to be the third type of business service; wherein, the third type of business service is the business service type in which customers actively seek help from customer service robots.

[0164] Specifically, based on the first target data, through data analysis, it was found that: the target customer's gaze was pointing in the direction of the customer service robot, and / or the target customer was moving closer to the customer service robot, etc., which can be used to determine that the target customer has a tendency to seek help from the customer service robot.

[0165] In some embodiments, the acquisition of second target data about the target customer according to the target processing rules may specifically include the following: collecting first image data and / or audio data and / or image data within a third time period following the first image data, as the second target data, according to the target processing rules. Specifically, the third time period may be five minutes. Of course, in specific implementations, other suitable durations may be set as the third time period, depending on the specific circumstances.

[0166] In some embodiments, the above-described determination of whether a target customer has a business service need by processing the second target data according to the target processing rules may include the following:

[0167] S1: Based on the second target data, confirm whether the target customer is seeking help from the customer service robot;

[0168] S2: If it is confirmed that the target customer is seeking help from the customer service robot, determine that the target customer has a business service need.

[0169] Specifically, for example, based on the second target data, if it is detected that the target customer is getting closer to the customer service robot and the distance between them is less than a preset distance threshold; and / or, if it is detected that the target customer is speaking towards the location of the customer service robot, it can be determined that the target customer really has a business service need.

[0170] In some embodiments, the above-mentioned interaction with the target customer according to preset interaction rules may include the following:

[0171] S1: Generate matching interactive text based on preset interaction rules;

[0172] S2: Generate corresponding interactive audio based on the interactive text;

[0173] S3: Play the interactive audio to the target customer.

[0174] Based on the above embodiments, when the customer service robot determines that the target customer really has a business service need, it can be automatically activated and generate matching interactive text. By playing interactive audio based on the interactive text to the target customer, it can actively interact with the target customer so that the target customer can obtain a better service experience.

[0175] In practice, for example, refer to Figure 5 As shown, speech recognition and image recognition can be performed on the second audio data and the second image data respectively to obtain the corresponding speech recognition results and image recognition results; then the speech recognition results and image recognition results can be combined to obtain the target recognition result; by using a preset matching model to process the target recognition result, the preset matching text can be selected from the preset text set as the interactive text.

[0176] Furthermore, corresponding interactive images (including interactive videos and / or interactive images) can be generated based on the interactive text; and then the interactive images can be played to the target customer.

[0177] In some embodiments, while playing the interactive audio to the target customer, the method may further include: flashing an indicator light at a matching frequency according to preset interaction rules to alert the target customer.

[0178] Based on the above embodiments, by flashing indicator lights at a matching frequency, target customers can more easily notice the customer service robot, thereby seeking help from the customer service robot more effectively and further improving the customer service experience.

[0179] As can be seen from the above, the data processing method provided in this embodiment involves a customer service robot monitoring ambient sound. When a preset triggering condition is met, the robot acquires and determines the current business service type based on first target data about the target customer. Then, based on the current business service type, it determines a matching target processing rule from preset processing rules. According to the target processing rule, it acquires and determines whether the target customer has a business service need based on second target data about the target customer. If the target customer has a business service need, the robot interacts with the target customer according to preset interaction rules to provide matching business services. By first differentiating different business service types based on the first target data, and then acquiring and utilizing the second target data according to the matching processing rules for different business server types, the robot accurately determines whether the target customer truly has a business service need. This effectively and comprehensively covers different business service scenarios, accurately detects and identifies customers with business service needs, and proactively interacts with customers without requiring them to be woken up, intelligently providing relevant business services and ensuring a better customer experience.

[0180] This specification also provides a customer service robot, including a processor and a memory for storing processor-executable instructions. Specifically, the processor can perform the following steps according to the instructions: monitoring ambient sound to determine if a preset triggering condition is met; if the preset triggering condition is met, acquiring first target data about the target customer; determining the current business service type based on the first target data; determining a matching target processing rule from preset processing rules based on the current business service type; acquiring second target data about the target customer based on the target processing rule; determining whether the target customer has a business service need by processing the second target data according to the target processing rule; and, if the target customer has a business service need, interacting with the target customer according to preset interaction rules to provide matching business services.

[0181] To execute the above instructions more accurately, please refer to... Figure 6 As shown in the embodiments of this specification, another specific customer service robot is also provided. The customer service robot may specifically include a network communication port 601, a processor 602, and a memory 603. The above structures are connected by internal cables so that the various structures can perform specific data interaction.

[0182] Specifically, the network communication port 601 can be used to receive startup commands.

[0183] The processor 602 can specifically be used to respond to a startup command, monitor ambient sound to determine whether a preset trigger condition is met; if the preset trigger condition is met, acquire first target data about the target customer; determine the current business service type based on the first target data; determine a matching target processing rule from preset processing rules based on the current business service type; acquire second target data about the target customer based on the target processing rule; determine whether the target customer has a business service need by processing the second target data according to the target processing rule; if the target customer has a business service need, interact with the target customer according to preset interaction rules to provide a matching business service to the target customer.

[0184] The memory 603 can be used to store the corresponding instruction program.

[0185] In this embodiment, the network communication port 601 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0186] In this embodiment, the processor 602 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0187] In this embodiment, the memory 603 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0188] This specification also provides a computer storage medium based on the above data processing method. The computer storage medium stores computer program instructions that, when executed, perform the following: monitor ambient sound to determine whether a preset trigger condition is met; if the preset trigger condition is met, acquire first target data about the target customer; determine the current business service type based on the first target data; determine a matching target processing rule from preset processing rules based on the current business service type; acquire second target data about the target customer based on the target processing rule; determine whether the target customer has a business service need by processing the second target data according to the target processing rule; and if the target customer has a business service need, interact with the target customer according to preset interaction rules to provide a matching business service.

[0189] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0190] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer storage medium can be explained in comparison with other implementation methods, and will not be repeated here.

[0191] See Figure 7 As shown, at the software level, this specification also provides a data processing apparatus, which may specifically include the following structural modules:

[0192] The monitoring module 701 can be used to monitor ambient sound to determine whether the preset triggering conditions are met.

[0193] The first acquisition module 702 can be used to acquire first target data about the target customer when it is determined that the preset triggering conditions are met.

[0194] The first determining module 703 can be specifically used to determine the current business service type based on the first target data;

[0195] The second determining module 704 can be used to determine the matching target processing rule from the preset processing rules based on the current business service type.

[0196] The second acquisition module 705 can be used to acquire second target data about the target customer according to the target processing rules;

[0197] The third determination module 706 can be used to determine whether a target customer has a business service need by processing the second target data according to the target processing rules.

[0198] The interaction module 707 can be used to interact with the target customer according to preset interaction rules when it is determined that the target user has business service needs, so as to provide the target customer with matching business services.

[0199] In some embodiments, when the monitoring module 701 is specifically implemented, it can monitor ambient sound in the following manner to determine whether the preset triggering conditions are met: collect the current ambient sound and perform human voice detection on the current ambient sound to determine whether there is a human voice in the current ambient sound; if it is determined that there is a human voice in the current ambient sound, extract the human voice audio from the current ambient sound; detect whether the human voice audio is interference sound; if it is determined that the human voice audio is not interference sound, determine that there is a target customer in the current environment and determine that the preset triggering conditions are met.

[0200] In some embodiments, the interference sound may include at least one of the following: broadcast sound, staff voice, meaningless human voice, etc.

[0201] In some embodiments, when the first determining module 703 is specifically implemented, it can determine the current business service type based on the first target data in the following manner: process the first audio data in the first target data using a preset speech recognition model to obtain the corresponding first text; detect whether there are preset keywords or related words related to the preset keywords in the first text; if it is determined that there are preset keywords or related words related to the preset keywords in the first text, determine that the current business service type is a first type of business service type; wherein, the first type of business service type is a business service type in which customers inquire about business issues and wait for a reply.

[0202] In some embodiments, when the second acquisition module 705 is specifically implemented, it can acquire second target data about the target customer according to the target processing rules in the following manner: according to the target processing rules, it collects audio data within a first time period after the first audio data as the second audio data in the second target data; and / or, according to the target processing rules, it uses the first audio data to determine the current location of the target customer; and according to a preset time interval, it takes multiple images toward the current location of the target customer as the second image data in the second target data.

[0203] In some embodiments, when the third determining module 706 is specifically implemented, it can determine whether a target customer has a business service need by processing the second target data according to the target processing rules in the following manner: using a preset speech recognition model to process the second audio data in the second target data to obtain the corresponding second text; detecting whether there is a reply text related to preset keywords or related words in the second text; and determining that the target customer has a business service need if the reply text is not found in the second text.

[0204] In some embodiments, when the first determining module 703 is specifically implemented, it can determine whether the target customer has a business service need by processing the second target data according to the target processing rules in the following manner: based on the second image data in the second target data, detect whether there are staff members at the current location of the target customer; if it is determined that there are no staff members at the current location of the target customer, determine that the target customer has a business service need.

[0205] In some embodiments, when the first determining module 703 is specifically implemented, it may also determine the current business service type based on the first target data in the following manner: based on the first image data in the first target data, detect whether the target customer is currently using a self-service machine to handle self-service business; if it is determined that the target customer is currently using a self-service machine to handle self-service business, determine the current business service type as the second type of business service; wherein, the second type of business service is the business service type in which the customer uses a self-service machine to handle self-service business.

[0206] In some embodiments, when the second acquisition module 705 is specifically implemented, it can acquire second target data about the target customer according to the target processing rules in the following manner: according to the target processing rules, video data within a second time period after the first image data is recorded in the direction of the self-service machine location, as the second image data in the second target data.

[0207] In some embodiments, when the third determining module 706 is specifically implemented, it can determine whether the target customer has a business service need by processing the second target data according to the target processing rules in the following manner: based on the second image data in the second target data, detect whether the target customer encounters a problem when using the self-service machine to handle self-service business; if it is determined that the target customer encounters a problem when using the self-service machine to handle self-service business, determine that the target customer has a business service need.

[0208] In some embodiments, when the interaction module 707 is specifically implemented, it can interact with the target customer in the following manner according to preset interaction rules: generate matching interaction text according to preset interaction rules; generate corresponding interaction audio according to interaction text; and play the interaction audio to the target customer.

[0209] In some embodiments, the interaction module 707, while playing the interactive audio to the target customer, can also be used to flash an indicator light at a matching frequency according to preset interaction rules to prompt the target customer.

[0210] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection between the devices or units shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0211] As can be seen from the above, the data processing apparatus provided in the embodiments of this specification first distinguishes different business service types based on the first target data; then, for different business server types, it acquires and utilizes the second target data according to the matching processing rules to accurately determine whether the target customer currently has a genuine business service need. This allows for more effective and comprehensive coverage of different business service scenarios, accurate detection and identification of customers with business service needs, and proactive interaction with customers without requiring customer activation, intelligently providing relevant business services and enabling customers to obtain a better service experience.

[0212] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0213] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0214] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A data processing method, characterized in that, Applications in customer service robots include: Monitor ambient sound to determine if the preset triggering conditions are met; If the preset triggering conditions are met, obtain the first target data about the target customer; Based on the first target data, the current business service type is determined; wherein, the business service type includes a first type of business service, a second type of business service, and a third type of business service; the first type of business service is the business service type where customers consult business issues and wait for a reply; the second type of business service is the business service type where customers use self-service machines to handle self-service business; the third type of business service is the business service type where customers actively seek help from customer service robots; Based on the current business service type, determine the matching target processing rule from the preset processing rules; Based on the target processing rules, obtain secondary target data about the target customer; Based on the target processing rules, the second target data is processed to determine whether the target customer has a business service need; Once it is determined that the target user has a need for business services, the system interacts with the target customer according to the preset interaction rules in order to provide the target customer with matching business services. This includes monitoring ambient sound to determine whether preset triggering conditions are met, including: Collect the current ambient sound and perform human voice detection on the current ambient sound to determine whether there is a human voice in the current ambient sound; If it is determined that there is human voice in the current ambient sound, extract the human voice audio from the current ambient sound. Detect whether the human voice audio is interference sound; If it is determined that the human voice audio is not interference, it is determined that a target customer exists in the current environment, and the preset triggering conditions are met. Specifically, based on the target processing rules, the second target data is processed to determine whether the target customer has business service needs, including: Based on the second image data in the second target data, detect whether the target customer encounters any problems when using the self-service machine to handle self-service business; If it is determined that the target customer has encountered problems when using the self-service machine to handle self-service business, it is determined that the target customer has a business service need; Specifically, based on the second image data in the second target data, it detects whether the target customer encounters problems when using the self-service machine to handle self-service business. This includes: analyzing the target customer's facial expression changes during the second time period based on the second image data to determine the target customer's emotional changes; and determining whether the target customer encounters problems when using the self-service machine to handle self-service business based on the target customer's emotional changes.

2. The method according to claim 1, characterized in that, The interference sounds include at least one of the following: broadcast sounds, staff voices, or meaningless human voices.

3. The method according to claim 1, characterized in that, Based on the first target data, determine the current business service type, including: The first audio data in the first target data is processed using a preset speech recognition model to obtain the corresponding first text; Detect whether the first text contains preset keywords or related words. If it is determined that there are preset keywords or related words in the first text, the current business service type is determined to be the first type of business service; wherein, the first type of business service is the business service type in which customers inquire about business issues and wait for a reply.

4. The method according to claim 3, characterized in that, Based on the target processing rules, obtain secondary target data about the target customer, including: According to the target processing rules, audio data within the first time period after the first audio data is collected is used as the second audio data in the second target data. And / or, According to the target processing rules, the current location of the target customer is determined using the first audio data; according to the preset time interval, multiple images are taken at the current location of the target customer, which are used as the second image data in the second target data.

5. The method according to claim 4, characterized in that, Based on the target processing rules, by processing the second target data, it is determined whether the target customer has business service needs, including: The second audio data in the second target data is processed using a preset speech recognition model to obtain the corresponding second text; Detect whether there is a response text in the second text that is related to preset keywords or related terms; If the response text is not found in the second text, it is determined that the target customer has a business service need.

6. The method according to claim 4, characterized in that, Based on the target processing rules, by processing the second target data, it is determined whether the target customer has business service needs, including: Based on the second image data in the second target data, detect whether there are staff members at the current location of the target customer; If no staff are present at the target customer's current location, it is determined that the target customer has a need for business services.

7. The method according to claim 1, characterized in that, Based on the first target data, determining the current business service type also includes: Based on the first image data in the first target data, detect whether the target customer is currently using a self-service machine to conduct self-service business; If it is determined that the target customer is currently using a self-service machine to conduct self-service business, the current business service type is determined to be the second type of business service; wherein, the second type of business service is the business service type in which the customer uses a self-service machine to conduct self-service business.

8. The method according to claim 7, characterized in that, Based on the target processing rules, obtain secondary target data about the target customer, including: According to the target processing rules, video data within a second time period following the first image data is recorded in the direction of the self-service machine, and this data is used as the second image data in the second target data.

9. The method according to claim 1, characterized in that, Interact with target customers according to preset interaction rules, including: Generate matching interactive text based on preset interaction rules; Generate corresponding interactive audio based on the interactive text; Play the interactive audio to the target customer.

10. The method according to claim 9, characterized in that, While playing the interactive audio to the target customer, the method further includes: According to preset interaction rules, the indicator light flashes at a matching frequency to alert the target customer.

11. A data processing apparatus, characterized in that, Applications in customer service robots include: The monitoring module is used to monitor ambient sound to determine whether the preset triggering conditions are met. The first acquisition module is used to acquire first target data about the target customer when it is determined that the preset triggering conditions are met. The first determining module is used to determine the current business service type based on the first target data; wherein, the business service type includes a first type of business service, a second type of business service, and a third type of business service; the first type of business service is the business service type in which customers consult business issues and wait for a reply; the second type of business service is the business service type in which customers use self-service machines to handle self-service business; and the third type of business service is the business service type in which customers actively seek help from customer service robots. The second determination module is used to determine the matching target processing rule from the preset processing rules based on the current business service type. The second acquisition module is used to acquire second target data about the target customer according to the target processing rules; The third determination module is used to determine whether the target customer has a business service need by processing the second target data according to the target processing rules. The interaction module is used to interact with the target customer according to preset interaction rules when it is determined that the target user has a business service need, so as to provide the target customer with matching business services. The monitoring module monitors ambient sound in the following manner to determine whether the preset triggering conditions are met: it collects the current ambient sound and performs human voice detection on the current ambient sound to determine whether there is a human voice in the current ambient sound; if it is determined that there is a human voice in the current ambient sound, it extracts the human voice audio from the current ambient sound; it detects whether the human voice audio is interference sound; if it is determined that the human voice audio is not interference sound, it determines that there is a target customer in the current environment and that the preset triggering conditions are met. The third determination module determines whether a target customer has a business service need by processing the second target data according to the target processing rules in the following way: based on the second image data in the second target data, it detects whether the target customer encounters problems when using the self-service machine to handle self-service business; if it is determined that the target customer encounters problems when using the self-service machine to handle self-service business, it determines that the target customer has a business service need. Specifically, based on the second image data in the second target data, it detects whether the target customer encounters problems when using the self-service machine to handle self-service business. This includes: analyzing the target customer's facial expression changes during the second time period based on the second image data to determine the target customer's emotional changes; and determining whether the target customer encounters problems when using the self-service machine to handle self-service business based on the target customer's emotional changes.

12. A customer service robot, characterized in that, It includes a processor and a memory for storing processor-executable instructions, wherein the processor, when executing the instructions, implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10.

Images