Call information processing method, electronic device, storage medium and computer program product

Abstract

The embodiment of the invention provides a call information processing method, electronic equipment, a storage medium and a computer program product. According to the embodiment of the invention, the method comprises the steps: carrying out the voice-to-text processing of voice call data recorded in a voice call process when an order placing user and an order receiving merchant process a household service order, and obtaining a to-be-recognized call text; according to the method, the to-be-recognized call texts are preliminarily screened by using the call text recognition rule, the call texts suspected to have illegal call behaviors are recalled, and illegal call recognition is continuously performed on the call texts suspected to have illegal call behaviors by using the large language model, so that the text quantity required to be processed by the large language model can be effectively reduced, and the recognition efficiency is improved. The use cost of the large language model is reduced, the recognition accuracy is improved, and a guarantee is provided for guaranteeing the service quality of a housekeeping service platform and / or user information safety.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a call information processing method, electronic device, storage medium and computer program product. Background Art

[0002] The housekeeping service platform is an online platform that provides consumers with diversified housekeeping services. It aims to simplify the booking and management process of housekeeping services, improve the efficiency and transparency of housekeeping services, and provide consumers with a more convenient and personalized service experience. Through the housekeeping service platform, consumers can easily obtain a variety of life services including cleaning, home appliance repair, moving, etc. to meet various needs in daily life.

[0003] In actual applications, consumers can place orders online through various user terminals such as applications (APPs) and mini-programs. The housekeeping service platform sends orders to the merchant's merchant terminal, and the merchant checks the order information through the merchant terminal and decides whether to accept the order. After the merchant accepts the order, it can arrange for staff to provide door-to-door service. In order to ensure service quality and user information security, consumers and merchants can communicate by phone through the virtual mobile phone number provided by the housekeeping service platform.

[0004] However, during the telephone communication process, there may be unreasonable calling behaviors, such as excessive language, malicious sales or privacy information leakage, etc. This situation may not only affect the service quality of the housekeeping service platform, but also cause user information security issues. Summary of the invention

[0005] Multiple aspects of the present application provide a call information processing method, an electronic device, a storage medium and a computer program product to improve the accuracy of identifying illegal call behaviors.

[0006] An embodiment of the present application provides a call information processing method, including: performing speech-to-text processing on voice call data to be identified to obtain call text to be identified, wherein the voice call data to be identified is voice call data recorded during a voice call between an ordering user and an order-taking merchant when processing a target housekeeping service order; using a call text recognition rule to identify whether the call text to be identified includes set call information, if the call text to be identified includes the set call information, determining that the call text to be identified is a suspected illegal call text; calling a large language model to perform illegal call recognition on the suspected illegal call text to obtain a recognition result, and the recognition result is used to indicate whether the voice call between the ordering user and the order-taking merchant is an illegal call with unreasonable call behavior.

[0007] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to execute the steps in the call information processing method.

[0008] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps in the call information processing method.

[0009] An embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the call information processing method.

[0010] In an embodiment of the present application, voice call data recorded during a voice call between an ordering user and an order-taking merchant when processing a housekeeping service order is subjected to voice-to-text processing to obtain call text to be identified; call text recognition rules are used to perform preliminary screening on the call text to be identified, and call texts suspected of violating regulations are recalled, and a large language model is used to continue to perform illegal call identification on call texts suspected of violating regulations. This can effectively reduce the amount of text that needs to be processed by the large language model, reduce the cost of using the large language model, and improve the recognition accuracy, thereby providing protection for the service quality and / or user information security of the housekeeping service platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 An exemplary application scenario diagram provided for this application example;

[0013] Figure 2 A flowchart of a call information processing method provided in an embodiment of the present application;

[0014] Figure 3 is an exemplary prompt word template;

[0015] Figure 4 A flowchart of another call information processing method provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0018] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B may be singular or plural. In the text description of the present application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In addition, in the embodiments of the present application, "first", "second", "third", etc. are only used to distinguish the contents of different objects and have no other special meanings.

[0019] Figure 1 An exemplary application scenario diagram provided for this application example. In the housekeeping service scenario, users can place orders online through various user terminals such as applications (APPs) and mini-programs. The housekeeping service platform dispatches orders to the merchant's merchant terminal, and the merchant checks the order information through the merchant terminal and decides whether to accept the order. After the merchant accepts the order, staff can be arranged for door-to-door service. In order to ensure service quality and user information security, it supports telephone communication between users and merchants through virtual mobile phone numbers provided by the housekeeping service platform. During the telephone communication process, users or merchants may ask for the other party's real contact information, such as a real mobile phone number or an instant messaging account. For another example, during the telephone communication process, both parties speak excessively or the merchant maliciously promotes. Various unreasonable call behaviors may not only affect the service quality of the housekeeping service platform, but may also cause user information security issues.

[0020] In actual applications, the housekeeping service platform can collect the voice call data generated by users and merchants during telephone communication, convert the voice call data into text, and obtain the call text to be identified. The housekeeping service platform first determines whether the call text to be identified is a suspected illegal call text based on the call text recognition rules, and then identifies the suspected illegal call text based on the large language model. The housekeeping service platform can also send the illegal call identification results to the merchant's merchant end to prompt the merchant to avoid illegal calls. Therefore, based on the call text recognition rules and the large language model, it is efficient and accurate to identify whether the user or merchant has illegal call behavior, which provides protection for ensuring the service quality and / or user information security of the housekeeping service platform.

[0021] It should be noted that Figure 1 The application scenario shown is only an exemplary application scenario, and the present application embodiment does not limit the application scenario. Figure 1 does not limit the equipment included in Figure 1 The positional relationship between the devices is limited.

[0022] Among them, the user end or the merchant end can be hardware or software. When the user end or the merchant end is hardware, the user end or the merchant end is, for example, a mobile phone, a tablet computer, a desktop computer, a wearable smart device, a smart home device, etc. When the user end or the merchant end is software, it can be installed in the hardware devices listed above. At this time, the user end or the merchant end is, for example, multiple software modules or a single software module, etc., and the embodiments of the present application are not limited. The housekeeping service platform can be hardware or software. When the housekeeping service platform is hardware, the housekeeping service platform is a distributed server cluster composed of a single server or multiple servers. When the housekeeping service platform is software, it can be multiple software modules or a single software module, etc., and the embodiments of the present application are not limited.

[0023] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following is a detailed description of the technical solutions provided by each embodiment of the present application in conjunction with the accompanying drawings.

[0024] Figure 2 A flowchart of a call information processing method provided in an embodiment of the present application. Figure 2 , the method may include the following steps:

[0025] 201. Perform voice-to-text processing on the voice call data to be identified to obtain the call text to be identified, wherein the voice call data to be identified is the voice call data recorded during the voice call between the ordering user and the order-taking merchant when processing the target housekeeping service order.

[0026] In actual applications, users can make online reservations for various housekeeping services on the housekeeping service platform. When users submit orders for housekeeping services on the housekeeping service platform, the housekeeping service platform can select merchants to quickly dispatch orders, and merchants accept orders on demand. For the sake of description and distinction, users are referred to as ordering users, merchants are referred to as order-receiving merchants, and the orders submitted by ordering users are referred to as target housekeeping service orders.

[0027] In actual applications, when the merchant accepts the order, the merchant and the ordering user can use the virtual mobile phone number provided by the housekeeping service platform to communicate by phone, and the virtual mobile phone number can effectively protect the user's privacy information. For example, before the merchant's staff comes to provide door-to-door service, both parties can use the virtual mobile phone number to confirm service details, agree on the door-to-door time, and communicate special requirements to ensure smooth service.

[0028] In this embodiment, voice recording is performed during a voice call between an ordering user and an order-taking merchant when processing a target housekeeping service order, and the recorded voice call data is used as voice call data to be recognized, and the voice call to be recognized is subjected to speech-to-text (Speech-to-Text, STT) processing to obtain the call text to be recognized.

[0029] 202. Use call text recognition rules to identify whether the call text to be recognized includes set call information. If the call text to be recognized includes the set call information, determine that the call text to be recognized is a suspected illegal call text.

[0030] In actual applications, the set call information is flexibly defined as needed. The set call information includes, but is not limited to: asking for real contact information, sensitive words, and promoting products or service information that does not belong to the housekeeping service platform. Real contact information includes, but is not limited to: real mobile phone numbers or instant messaging accounts. Sensitive words include, for example, dirty words, inappropriate words, etc. Promotion of products or service information that does not belong to the housekeeping service platform includes, for example, malicious promotion information, advertisements, etc.

[0031] Specifically, the call text recognition rule refers to a rule for effectively identifying the call text containing the set call information, and the call text recognition rule can be flexibly defined as needed. Further optionally, in order to improve the recognition accuracy of illegal call behavior, the call text recognition rule includes: multiple specified keywords associated with the set call information, and / or, a specified string screening condition, the specified string screening condition includes: a condition on the number of consecutive numbers in the specified string and a condition on the total number of numbers in the specified string; accordingly, the implementation method of using the call text recognition rule to identify whether the call text to be recognized includes the set call information is: matching the call text to be recognized with multiple specified keywords in the call text recognition rule, if the call text to be recognized includes the specified keywords, then determining that the call text to be recognized includes the call information related to the setting; and / or, using the specified string screening condition in the call text recognition rule to screen the call text to be recognized, if a specified string that meets the specified string screening condition is screened out from the call text to be recognized, then determining that the call text to be recognized includes the set call information.

[0032] Specifically, the call text recognition rule may include a regular expression for matching multiple designated keywords associated with the set call information. The regular expression in the call text recognition rule may be used to identify whether the call text to be recognized includes the set call information.

[0033] Taking the illegal call as an example of a voice call requesting real contact information, the specified keywords associated with the set call information include, but are not limited to: "WeChat", "number", "mobile number", "contact information", "phone number", "leave a phone number", "real phone number", "virtual number", "virtual mobile number", "real mobile number", "mobile number", "real landline number". The regular expression is, for example, "WeChat|number|mobile number|contact information|phone number|leave a phone number|real phone number|virtual number".

[0034] Taking the illegal call as a voice call involving sensitive words as an example, the designated keywords associated with the set call information are various sensitive words.

[0035] Taking illegal calls as voice calls involving the promotion of goods or service information that do not belong to the housekeeping service platform as an example, the designated keywords associated with the set call information include, but are not limited to: purchase, discount, special offer, discount, recommendation, financial management, promotion, etc.

[0036] In actual applications, some application scenarios may prohibit the appearance of sensitive information related to numbers during voice calls. For example, real mobile phone numbers, bank card numbers, ID card numbers, etc. Based on this, the call text recognition rule can also use the specified string screening conditions to detect continuous numbers and the total number of numbers. The number of continuous numbers in the specified string meets the corresponding number condition, and the total number of numbers in the specified string meets the corresponding total number condition. The number condition of continuous numbers in the specified string is that the number of continuous numbers in the specified string is greater than or equal to the first number; the total number condition of numbers in the specified string is that the total number of numbers in the specified string is greater than or equal to the second number, the first number is less than the second number, and the first number and the second number are flexibly set as needed. For example, the first number is 3, and the second number is 6. In this way, the number condition of continuous numbers in the specified string is that the number of continuous numbers in the specified string is greater than or equal to 3; the total number condition of numbers in the specified string is that the total number of numbers in the specified string is greater than or equal to 6.

[0037] In practical applications, due to various reasons, there may be errors in the text conversion results of speech-to-text. For example, the numbers in the voice call data to be recognized may be misrecognized as non-numerical characters such as Chinese characters or English characters. Specifically, for example, the user placing an order or the merchant receiving the order may communicate in a dialect or have an inaccurate pronunciation, which may affect the accuracy of speech-to-text conversion. Or, the low precision of the speech-to-text algorithm may result in errors in the text conversion results. Specifically, for example, the call text to be recognized includes multiple strings such as "123 one zero zero one 1234", "one 23 one zero zero one 1234", "123 one zero zero one zero zero zero zero", etc. Among them, "123 one zero zero one 1234" is recognized as the specified string. The "one" in the call text to be recognized should be the number 1, and the "zero" in the call text to be recognized should be the number 0.

[0038] Therefore, both the number of consecutive digits in the specified string and the total number of digits in the specified string are required, which can effectively handle the situation where there are errors in the text conversion results of speech-to-text and improve the recognition accuracy of identifying illegal calls using call text recognition rules.

[0039] In this embodiment, if it is recognized by the call text recognition rule that the call text to be recognized includes the set call information, it is determined that the call text to be recognized is a suspected illegal call text. A suspected illegal call text is also a call text that is suspected of having an illegal call behavior (which can also be called an unreasonable call behavior); if it is recognized by the call text recognition rule that the call text to be recognized does not include the set call information, it is determined that the call text to be recognized is not a suspected illegal call text, that is, it is determined that the call text to be recognized is a normal call text. A normal call text indicates that the user placing the order and the merchant receiving the order do not involve the set call information during the voice call.

[0040] 203. Invoke a large language model to identify illegal calls for the suspected illegal call text, and obtain an identification result. The identification result is used to indicate whether the voice call between the user placing the order and the merchant receiving the order is an illegal call with an unreasonable call behavior.

[0041] In practical applications, due to various reasons, the recognition accuracy of call text recognition rules in identifying illegal call behaviors is generally average. For example, since there may be noise, background sounds, etc. in the call recording, which may lead to errors in the text conversion results of speech-to-text, this will also affect the recognition accuracy of call text recognition rules. Another example is that due to the complex call recording conversation, there will be unexpected situations: such as mentioning the order number, access control password, or the merchant confirming the first and last digits of the user's mobile phone number with the user to determine the user's order during the call. This will also cause misjudgment of call text recognition rules.

[0042] Based on the above, call text recognition rules are used for initial screening, call texts suspected of violating regulations are recalled, and the large language model is used to continue to identify call texts suspected of violating regulations. This can effectively reduce the amount of text that needs to be processed by the large language model, reduce the cost of using the large language model, and improve recognition accuracy.

[0043] Among them, the large language model is also called the Large Language Model (LLM), which refers to a type of natural language processing model with a very large number of parameters. Large language models are usually based on deep learning architectures, especially the Transformer architecture. They learn the complex structure and rich contextual information of the language through pre-training on massive text data. Using large language models to identify illegal call behaviors can improve the accuracy of identifying illegal call behaviors.

[0044] In practical applications, the prompt word template corresponding to the large language model can be flexibly set. The suspected illegal call text is filled into the prompt word template as the target conversation information between the ordering user and the order-taking merchant to obtain the target prompt word, and the large language model is called to perform illegal call recognition on the suspected illegal call text based on the target prompt word to obtain the recognition result.

[0045] Further optionally, in order to improve the recognition accuracy of illegal call behavior, a large language model is called to identify illegal calls on suspected illegal call texts, and the implementation method for obtaining the recognition result is as follows: obtaining a prompt word template corresponding to the large language model, the prompt word template including: task description information, requirement description information and dialogue information filling items; wherein the task description information is used to guide the large language model to answer the target question according to the target dialogue information and the requirement description information, and the target question is to confirm whether the target dialogue information involves the set call information; the requirement description information is used to describe the answer selection requirements of the target question, and the answer selection requirements indicate that one is selected from multiple optional identification tags to answer the target question, and the selected identification tag for answering the target question is used as the recognition result, and the multiple optional identification tags include: an abnormal tag indicating unreasonable call behavior, a normal tag indicating reasonable call behavior, and an uncertain tag indicating whether the call behavior is reasonable; the suspected illegal call text is filled into the dialogue information filling item as the target dialogue information between the ordering user and the ordering merchant to obtain the target prompt word; the large language model is called to select an identification tag from multiple optional identification tags based on the target prompt word to answer the target question, and the selected identification tag for answering the target question is used as the recognition result.

[0046] Further optionally, in order to improve the recognition accuracy of illegal call behavior, the prompt word template also includes: knowledge base information, the knowledge base information includes: a mapping relationship between multiple different semantic understanding results and multiple different recognition results, the semantic understanding result reflects whether the set call information is involved; accordingly, calling the large language model to select an identification tag from multiple optional identification tags based on the target prompt word to answer the target question, and using the selected identification tag for answering the target question as the recognition result. The implementation method includes: calling the large language model to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain the target semantic understanding result; combining the target semantic understanding result and the knowledge base information to select an identification tag from multiple optional identification tags to answer the target question, and using the selected identification tag for answering the target question as the recognition result.

[0047] For example, the voice understanding result is "the merchant or user asks to add WeChat or asks for a mobile phone number, and the merchant or user speaks the full mobile phone number", and the recognition result is "determined to be an illegal call behavior to ask for user information."

[0048] For example, the voice understanding result is "the house number, specific address, order number, password, verification code and other numbers replied by the user are not complete mobile phone numbers", and the recognition result is "it is confirmed that this is not an illegal call behavior to ask for user information."

[0049] For example, the voice understanding result is "the merchant confirms the last four digits, the last digit, the first four digits and other parts of the mobile phone number with the user, not the complete mobile phone number", and the recognition result is "it is confirmed that this is not an illegal call behavior to ask for user information."

[0050] Further optionally, in order to improve the recognition accuracy of illegal call behavior, the prompt word template also includes: a reference case list, the reference case list includes multiple different reference cases, and the reference cases include: reference call text and its corresponding identification label; accordingly, calling the large language model to select an identification label from multiple optional identification labels based on the target prompt word to answer the target question, and using the selected identification label for answering the target question as the recognition result. The implementation method includes: calling the large language model to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain the target semantic understanding result; combining the target semantic understanding result, the knowledge base information answer and the reference case list to select an identification label from multiple optional identification labels to answer the target question, and using the selected identification label for answering the target question as the recognition result.

[0051] In practical applications, a manual feedback mechanism can be introduced for the recognition results output by the large language model. The manual feedback mechanism can not only correct the false positives or negatives of the large language model, but also help the large language model learn more complex scenarios and edge cases, thereby continuously improving its recognition ability.

[0052] Based on the above, further optionally, in order to improve the accuracy of identifying illegal call behaviors, if the identification result is used to indicate that the voice call between the ordering user and the order-taking merchant is an illegal call involving unreasonable call behaviors, illegal call notification information including the identification result is pushed to the merchant end of the order-taking merchant, so that the order-taking merchant can decide whether to initiate an appeal request, and the appeal request is used to indicate that the identification result is an error; in response to receiving the approval information of the appeal request from the manual customer service to the order-taking merchant, the identification tag of the call text to be identified and the recognition result configured by the manual customer service is added to the reference case list as a new reference case.

[0053] Specifically, when the large language model identifies that there is an illegal call between the ordering user and the ordering merchant, the illegal call notification information is automatically pushed to the merchant end of the ordering merchant. Illegal call notification information includes, but is not limited to: recognition results, call recordings (i.e., voice call data) or call text. If the housekeeping service platform will punish the ordering merchant, the penalty information can also be added to the illegal call notification information. The merchant initiates a complaint request through the merchant end, stating the reason why it believes that the recognition result is wrong, and provides relevant evidence (such as call background, explanation, etc.). The manual customer service team of the home appliance service platform reviews the complaint request of the ordering merchant. The manual customer service will determine whether the recognition result is correct based on the evidence and call records provided. If the customer service confirms that the recognition result is indeed wrong, the recognition result output by the large language model will be overturned, and the voice call between the ordering user and the ordering merchant will be marked as "non-violation". If the customer service confirms that the recognition result is correct, the recognition result output by the large language model will be maintained and the merchant will be notified.

[0054] In actual applications, after the manual customer service approves the call text that was incorrectly recognized by the large language model, the manual customer service will independently configure the correct identification label corresponding to the recognition result of the call text, and add the call text and the identification label of the recognition result configured by the manual customer service as a new reference case to the reference case list. The new reference case can be used for subsequent training and optimization of the large language model, helping the large language model to better understand and identify similar scenarios, learn more about the characteristics of reasonable and unreasonable call behaviors, and thus improve the recognition accuracy of illegal calls.

[0055] Further optionally, in order to improve the recognition accuracy of illegal call behavior, multiple call texts corresponding to the approval of the appeal request are periodically obtained, and the multiple call texts corresponding to the approval are analyzed to obtain new knowledge base information, and the new knowledge base information is added to the knowledge base information in the prompt word template.

[0056] In practical applications, a reasonable cycle (such as hourly, daily, weekly, monthly or quarterly) can be defined according to the business volume and data update frequency of the housekeeping service platform, so as to capture the latest illegal call behaviors in a timely manner. From all submitted appeal requests, screen out the appeal requests that have been approved by manual customer service. For the multiple call texts corresponding to these approved appeal requests, perform semantic understanding on these call texts to obtain the semantic understanding results corresponding to each of these call texts, and determine the identification tags corresponding to each of these call texts. Collect the semantic understanding results and identification tags corresponding to each of these call texts, and add them as newly added knowledge base information to the knowledge base information in the prompt word template to expand the knowledge base information in the prompt word template, thereby improving the recognition accuracy of illegal call behaviors of the large language model.

[0057] For example, the housekeeping service platform prohibits users and businesses from asking for real mobile phone numbers during voice calls. If a real mobile phone number is asked for during a voice call, the voice call will be determined to be an illegal call asking for user information. If it is determined to be an illegal call, the corresponding business will be punished, thereby improving the compliance of the business.

[0058] See also Figure 3 ,The prompt word template includes: task description information, requirement description information, ,knowledge base information, reference case list and dialogue information filling items.

[0059] The task description information is: Based on the conversation information between the merchant and the user, the knowledge base information, the reference cases and the requirement description information, answer the following question: Is it a request for user information?

[0060] The required description information is: "Output: yes, no, don't know". Among them, "yes" corresponds to an abnormal label indicating unreasonable call behavior; "no" corresponds to a normal label indicating reasonable call behavior; "don't know" corresponds to an uncertain label indicating whether the call behavior is reasonable.

[0061] The knowledge base information includes one or more of the following knowledge information:

[0062] 1. If a merchant or user asks to add WeChat or asks for a mobile phone number, and the merchant or user says the full mobile phone number, it will be determined as an illegal call requesting user information.

[0063] 2. If the house number, specific address, order number, password, verification code and other numbers replied by the user are not complete mobile phone numbers, it is determined that it is not an illegal call requesting user information.

[0064] 3. The merchant confirms with the user the last four digits, the last digit, the first four digits and other partial numbers of the mobile phone number. If it is not the complete mobile phone number, it is confirmed that it is not an illegal call to request user information.

[0065] The reference case list includes the following reference cases:

[0066] Reference Case 1 For the call text "The merchant only mentioned part of the mobile phone number, such as confirming the first and last numbers with the user", output: No;

[0067] Reference Case 2 For the call text "If the merchant asks for the house number, specific address, order number, password, verification code, and the user answers with numbers such as house number, specific address, order number, password, verification code, etc.", output: No

[0068] Reference Case 3: For the call text "If the merchant says that the user's contact information cannot be left, including WeChat and mobile phone numbers", output: No

[0069] Reference Case 4 For the call text "If the user confirms with the merchant whether it is this mobile number, but the merchant says that the mobile number is a virtual number", output: No

[0070] Reference Case 5: For the call text where "the merchant or user explicitly mentioned leaving contact information and gave a detailed mobile phone number or WeChat ID", the output is: Yes

[0071] Reference Case 6: For the call text "The merchant or user mentioned a number, etc., but it is not certain whether it is a mobile phone number or WeChat account", the output is: Uncertain

[0072] Fill in the following dialogue information in the dialogue information filling item: "User: Hello, hello, Merchant: Hey, I am a home maintenance merchant, what's your situation? User: Please wait a moment, let my wife tell you. Merchant: Yes, OK, User: Hello, hello, Merchant: Hey, hello, User: Hey, we have a house for rent now, Merchant: Is this the toilet? User: The toilet and the basin need to be replaced with new ones. Merchant: Oh, both need to be replaced with new ones. User: How about this? Let's consider it. OK, we will go there every day. Well, I'll add you on WeChat, and then I'll take a photo of the situation on the scene and show it to you. Merchant: Okay, you can, tell me your number. User: It's this number, you can add it and search it on WeChat. Merchant: This is a virtual number. User: Yes, there will be a virtual number, right? Merchant: Yes, yes, tell me your number. User: Uh, Manager An, um, 1860, 1919553. Merchant: 1860, 1919, um, ah, okay, I'll add you, um, okay, okay. User: Then I'll contact you tomorrow. Merchant: OK".

[0073] Specifically, for the conversation information, the recognition result output by the large language model is "JINGXUAN_WEIXIU_USER_INFORMATION", which indicates that the request for user information has been passed in violation of regulations.

[0074] For the voice call information that requests user information, the merchant corresponding to the order will be fined. The merchant can appeal within 24 hours after receiving the penalty, and the customer service staff will review the appeal. If the order does have any violations, the merchant's appeal will be rejected. If the order has no violations, the appeal will be approved. The voice call information that has been approved means that the recognition result is wrong. At this time, the voice call information with incorrect recognition results can be summarized regularly. If it is a common error situation, it will be summarized as new knowledge base information and added to the knowledge base information of the prompt word template of the large language model; if it is an individual case, it will be added to the reference case in the prompt word template of the large language model to avoid the re-misjudgment of such voice call information. For example, after a period of time online, it is found that merchants will fight against the penalty mechanism, and there will be a situation where merchants and users confirm the middle four digits of the mobile phone number. In this way, the knowledge of confirming the middle four digits will be added to the knowledge base, and such situations can be identified.

[0075] The technical solution provided in the embodiment of the present application performs speech-to-text processing on the voice call data recorded during the voice call between the ordering user and the order-taking merchant when processing the housekeeping service order, so as to obtain the call text to be identified; uses the call text recognition rules to perform preliminary screening on the call text to be identified, recalls the call texts suspected of violating the regulations, and uses the large language model to continue to identify the call texts suspected of violating the regulations. This can effectively reduce the amount of text that needs to be processed by the large language model, reduce the cost of using the large language model, and improve the recognition accuracy, so as to provide protection for ensuring the service quality and / or user information security of the housekeeping service platform.

[0076] For ease of understanding, the following Figure 4 See Figure 4 First, when the ordering user and the ordering merchant communicate by phone based on the virtual number provided by the housekeeping service platform, the call is recorded and the call recording text is obtained. Then, the call recording text is matched according to the call text recognition rules. If the call recording text is not a suspected illegal call text ( Figure 4 N branches in the ), the result output operation is executed, that is, the recognition result is output as not a violation call. If the call recording text is a suspected violation call text ( Figure 4The Y branch in the example above performs a large model recognition operation, that is, uses the large language model to perform a second illegal call recognition on the suspected illegal call text. If the output result of the large language model is an illegal call, the merchant who received the order will be punished. The correction module can also be used to correct the large language model based on the merchant's feedback information to improve the recognition accuracy of the large language model.

[0077] Traditional NLP (Natural Language Processing) models need to be trained according to business scenarios, which requires manual labeling of a certain amount of sample data, and the large language model technology effectively solves this problem. However, for the housekeeping service business, the call voice volume is large and the business content is complicated. The direct use of the large language model has problems such as high cost and low accuracy. The combination of rule-based initial screening and large language model recognition greatly reduces the cost of using the large language model and improves the recognition accuracy of illegal calls, thereby standardizing the actions of merchants contacting users and improving the compliance of contracts. Compared with the traditional NLP model, the use of the large language model greatly reduces the training cost of the model, and the operation is simple. You only need to modify the prompt word. According to the feedback data, the knowledge base information or reference case of the prompt word can be automatically used as the input of the large language model, which greatly reduces the cost of traditional NLP (model sample labeling, retraining, etc.).

[0078] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 203 can be device A; for another example, the execution subject of steps 201 and 202 can be device A, and the execution subject of step 203 can be device B; and so on.

[0079] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0080] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0081] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device includes: a memory 51 and a processor 52;

[0082] The memory 51 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0083] The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0084] The processor 52 is coupled to the memory 51 and is used to execute the computer program in the memory 51, so as to: perform speech-to-text processing on the voice call data to be identified to obtain the call text to be identified, wherein the voice call data to be identified is the voice call data recorded during the voice call between the ordering user and the order-taking merchant when processing the target housekeeping service order; use the call text recognition rule to identify whether the call text to be identified includes the set call information, if the call text to be identified includes the set call information, then determine that the call text to be identified is a suspected illegal call text; call the large language model to perform illegal call recognition on the suspected illegal call text to obtain a recognition result, and the recognition result is used to indicate whether the voice call between the ordering user and the order-taking merchant is an illegal call with unreasonable call behavior.

[0085] Optionally, the processor 52 calls the large language model to perform illegal call identification on the suspected illegal call text, and when the identification result is obtained, it is specifically used to: obtain a prompt word template corresponding to the large language model, the prompt word template includes: task description information, requirement description information and dialogue information filling item; wherein the task description information is used to guide the large language model to answer the target question according to the target dialogue information and the requirement description information, and the target question is to confirm whether the target dialogue information involves the set call information; the requirement description information is used to describe the answer selection requirement of the target question, and the answer selection requirement indicates to select one from multiple optional identification tags to answer the target question, and the selected identification tag for answering the target question is used as the recognition result, and the multiple optional identification tags include: an abnormal tag indicating unreasonable call behavior, a normal tag indicating reasonable call behavior, and an uncertain tag indicating whether the call behavior is reasonable; the suspected illegal call text is used as the target dialogue information between the ordering user and the ordering merchant to fill in the dialogue information filling item to obtain the target prompt word; call the large language model to select an identification tag from multiple optional identification tags based on the target prompt word to answer the target question, and the selected identification tag for answering the target question is used as the recognition result.

[0086] Optionally, the prompt word template also includes: knowledge base information, the knowledge base information includes: a mapping relationship between multiple different semantic understanding results and multiple different recognition results, the semantic understanding result reflects whether the set call information is involved; accordingly, the processor 52 calls the large language model to select an identification tag from multiple optional identification tags based on the target prompt word to answer the target question, and uses the identification tag selected to answer the target question as the recognition result, which is specifically used for: calling the large language model to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain the target semantic understanding result; combining the target semantic understanding result and the knowledge base information to select an identification tag from multiple optional identification tags to answer the target question, and uses the identification tag selected to answer the target question as the recognition result.

[0087] Optionally, the prompt word template also includes: a reference case list, the reference case list includes multiple different reference cases, and the reference cases include: reference call text and its corresponding identification label; accordingly, when the processor 52 calls the large language model to select an identification label from multiple optional identification labels based on the target prompt word to answer the target question, and uses the identification label selected to answer the target question as the recognition result, it is specifically used to: call the large language model to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain the target semantic understanding result; select an identification label from multiple optional identification labels to answer the target question in combination with the target semantic understanding result, the knowledge base information answer and the reference case list, and use the identification label selected to answer the target question as the recognition result.

[0088] Optionally, the processor 52 is also used for: if the recognition result is used to indicate that the voice call between the ordering user and the order-taking merchant is an illegal call involving unreasonable calling behavior, then pushing illegal call notification information including the recognition result to the merchant end of the order-taking merchant, so that the order-taking merchant can decide whether to initiate a complaint request, and the complaint request is used to indicate that the recognition result is incorrect; in response to receiving approval information of the complaint request from the manual customer service to the order-taking merchant, adding the call text to be identified and the recognition result identification label configured by the manual customer service as a new reference case to the reference case list.

[0089] Optionally, the processor 52 is also used to: periodically obtain multiple call texts to be identified corresponding to the approval of the appeal request, analyze the multiple call texts to be identified corresponding to the approval, obtain new knowledge base information, and add the new knowledge base information to the knowledge base information in the prompt word template.

[0090] Optionally, the call text recognition rules include: multiple specified keywords associated with the set call information, and / or, specified string filtering conditions, the specified string filtering conditions include: the number of consecutive numbers in the specified string and the total number of numbers in the specified string; accordingly, when the processor 52 uses the call text recognition rules to identify whether the call text to be recognized includes the set call information, it is specifically used to: match the call text to be recognized with multiple specified keywords in the call text recognition rules, if the call text to be recognized includes the specified keywords, it is determined that the call text to be recognized includes the set call information; and / or, use the specified string filtering conditions in the call text recognition rules to filter the call text to be recognized, if the specified string that meets the specified string filtering conditions is filtered out from the call text to be recognized, it is determined that the call text to be recognized includes the set call information.

[0091] Further, if Figure 5 As shown, the electronic device also includes: a communication component 53, a display 55, a power component 55, an audio component 56 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 In addition, Figure 5 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT (Internet of Things) device, or a server device such as a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 5If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 5 Components within the dashed box.

[0092] For the detailed implementation process of the processor executing each action, please refer to the relevant description in the aforementioned method embodiment or device embodiment, which will not be repeated here.

[0093] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0094] Accordingly, an embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement each step in the above method embodiment that can be executed by an electronic device.

[0095] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi (Wireless Fidelity), 2G (2 Generation, 2 Generation), 3G (3 Generation, 3 Generation), 4G (4 Generation, 4 Generation) / LTE (long Term Evolution), 5G (5 Generation, 5 Generation) and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (Near Field Communication, NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0096] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0097] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0098] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0104] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0105] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Phase Change RAM (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A call information processing method, characterized in that: include: Performing voice-to-text processing on the voice call data to be identified to obtain the call text to be identified, wherein the voice call data to be identified is the voice call data recorded during the voice call between the ordering user and the order-receiving merchant when processing the target housekeeping service order; Using a call text recognition rule to identify whether the call text to be recognized includes the set call information, if the call text to be recognized includes the set call information, determining that the call text to be recognized is a suspected illegal call text; The large language model is called to perform illegal call identification on the suspected illegal call text to obtain an identification result, wherein the identification result is used to indicate whether the voice call between the ordering user and the order-taking merchant is an illegal call involving unreasonable call behavior.

2. The method according to claim 1, characterized in that: The large language model is called to identify the suspected illegal call text, and the identification results include: Obtain a prompt word template corresponding to the large language model, the prompt word template including: task description information, requirement description information and dialogue information filling items; wherein the task description information is used to guide the large language model to answer a target question according to the target dialogue information and the requirement description information, the target question being to confirm whether the target dialogue information involves set call information; the requirement description information is used to describe an answer selection requirement for the target question, the answer selection requirement indicating that one of a plurality of optional identification tags is selected to answer the target question, and the selected identification tag for answering the target question is used as the recognition result, the plurality of optional identification tags including: an abnormal tag indicating unreasonable call behavior, a normal tag indicating reasonable call behavior and an uncertain tag indicating whether the uncertain call behavior is reasonable; Fill the suspected illegal call text into the dialogue information filling item as the target dialogue information between the ordering user and the order-taking merchant to obtain a target prompt word; The large language model is called to select an identification tag from a plurality of optional identification tags based on the target prompt word to answer the target question, and the identification tag selected for answering the target question is used as the recognition result.

3. The method according to claim 2, characterized in that The prompt word template further includes: knowledge base information, the knowledge base information including: a mapping relationship between a plurality of different semantic understanding results and a plurality of different recognition results, the semantic understanding result reflecting whether it involves set call information; Accordingly, calling the large language model to select an identification tag from a plurality of optional identification tags to answer the target question based on the target prompt word, and using the identification tag selected for answering the target question as the recognition result, includes: The large language model is called to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain a target semantic understanding result; an identification tag is selected from multiple optional identification tags to answer the target question in combination with the target semantic understanding result and the knowledge base information, and the identification tag selected for answering the target question is used as the recognition result.

4. The method according to claim 3, characterized in that The prompt word template further includes: a reference case list, the reference case list includes a plurality of different reference cases, the reference cases include: a reference call text and its corresponding identification tag; Accordingly, calling the large language model to select an identification tag from a plurality of optional identification tags to answer the target question based on the target prompt word, and using the identification tag selected for answering the target question as the recognition result, includes: The large language model is called to perform semantic understanding on the suspected illegal call text based on the target prompt word to obtain a target semantic understanding result; an identification tag is selected from multiple optional identification tags to answer the target question in combination with the target semantic understanding result, the knowledge base information answer and the reference case list, and the identification tag selected for answering the target question is used as the recognition result.

5. The method according to claim 4, characterized in that Also includes: If the recognition result is used to indicate that the voice call between the ordering user and the order-taking merchant is an illegal call involving unreasonable calling behavior, then push illegal call notification information including the recognition result to the merchant end of the order-taking merchant, so that the order-taking merchant can decide whether to initiate a complaint request, and the complaint request is used to indicate that the recognition result is an error; In response to receiving the approval information of the complaint request from the manual customer service to the merchant who accepted the order, the identification tag of the call text to be identified and the identification result configured by the manual customer service is added to the reference case list as a new reference case.

6. The method according to claim 5, characterized in that Also includes: Periodically obtain multiple conversation texts corresponding to the approval of the appeal request, analyze the multiple conversation texts corresponding to the approval, obtain newly added knowledge base information, and add the newly added knowledge base information to the knowledge base information in the prompt word template.

7. The method according to any one of claims 1 to 6, characterized in that: The call text recognition rule includes: a plurality of designated keywords associated with the set call information, and / or a designated string screening condition, wherein the designated string screening condition includes: a condition on the number of consecutive digits in the designated string and a condition on the total number of digits in the designated string; Accordingly, the call text recognition rule is used to identify whether the call text to be recognized includes the set call information, including: Matching the call text to be identified with a plurality of designated keywords in the call text identification rule, and if the call text to be identified includes the designated keywords, determining that the call text to be identified includes call information related to the setting; and / or, The call text to be identified is screened using the specified string screening condition in the call text identification rule. If a specified string that meets the specified string screening condition is screened out from the call text to be identified, it is determined that the call text to be identified includes the set call information.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is coupled to the memory and configured to execute the computer program to perform the steps in the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps in the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 7.

Images