A method and apparatus for responding to power business services

Abstract

The application discloses a power business service-oriented response method and device, and relates to the technical field of voice signal processing. The method comprises the following steps: collecting voice data of power customers in a power business hall through a smart voice recognition device worn by a staff of the power business hall and sending the voice data to a power service cloud system; extracting features of the voice data of the power customers through a shared feature extraction network to determine a customer emotional state; if the customer emotional state meets a preset special emotional state, controlling one or more video collection devices in the power business hall to collect video data of an environment where the power customers are located; inputting the video data, the voice data and the customer emotional state into a preset power marketing knowledge graph, and reasoning according to a cause-effect correlation relationship in the power marketing knowledge graph to determine an emotional cause corresponding to the customer emotional state and a corresponding business demand, and displaying the corresponding solution in the form of voice, video or text.

Description

Technical Field

[0001] This application relates to the field of voice signal processing technology, and in particular to a response method and device for electricity business services. Background Technology

[0002] Electricity sales services are highly specialized, time-sensitive, and complexly interconnected. Although intelligent customer service systems are now widely used, enabling automatic responses through pre-set question and answer databases, customer questions in electricity sales services often involve specialized electrical knowledge, such as unstable power supply, which causes significant problems for some electricity users or businesses, and issues like electricity pricing. When customers visit the sales office to inquire about problems such as power outages, unstable power supply, or abnormal electricity bills, they sometimes experience strong negative emotions. This necessitates that electricity sales office staff provide professional, rapid, and accurate answers.

[0003] Relying solely on the professional knowledge of electricity service hall staff requires a high level of experience and often necessitates manual data retrieval based on information provided by users, which is inefficient and can easily escalate conflicts when customers are emotionally agitated. Furthermore, existing systems, which primarily employ question-and-answer responses, often fail to accurately pinpoint the problem and require further manual investigation.

[0004] Therefore, for the electricity marketing scenario, how to trace back from voice, emotion, environment to business root causes has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a response method and device for electricity business services to solve the following technical problem: how to trace back from voice, emotion, environment to business root causes.

[0006] In a first aspect, embodiments of this application provide a response method for electricity business services. The method includes: collecting voice data of electricity customers in the business hall through an intelligent voice recognition device worn by staff in the business hall, and sending the voice data of electricity customers to an electricity service cloud system; the electricity service cloud system extracts features from the voice data of electricity customers through a preset shared feature extraction network to determine the customer's emotional state; if the customer's emotional state matches a preset special emotional state, controlling one or more video acquisition devices in the electricity business hall to collect video data of the environment in which the electricity customer is located; the electricity service cloud system inputs the video data, voice data, and customer emotional state into a preset electricity marketing knowledge graph, and infers based on the causal relationships in the electricity marketing knowledge graph to determine the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, and displays the corresponding solutions in the form of voice, video, or text.

[0007] In one implementation of this application, voice data of electricity customers in the service hall is collected using a smart voice recognition device worn by the staff. Specifically, this includes: real-time acquisition of mixed voice signals from interactions between the staff and customers using a multi-channel microphone worn by the staff; filtering out voice segments in the mixed voice signal whose voiceprints have a similarity to the staff's voiceprints reaching a preset threshold, identifying these segments as the staff's voice, and marking unmatched voice segments in the mixed voice signal; wherein, unmatched voice segments include customer voice and interference noise; and filtering the unmatched voice segments according to the service hall's proprietary interference feature library, removing those containing self-service payment machine prompts, etc. Interference signals in the power service hall scene, including the buzzing of power terminals and the announcements from the queuing machine, are filtered to retain candidate segments of power customer voices with human voice characteristics. By analyzing the voice timing, business semantic logic, and the voices of power service hall staff, the correlation between the voices of power service hall staff and customers in business interactions is confirmed. Voices of other irrelevant personnel in the power service hall are removed from the candidate segments of power customer voices to obtain the power customer voice signal. For multiple voice segments of the same power customer, multiple different encoding modes are used. Clear and coherent voice segments are encoded with low bit rate. Non-important voice segments are encoded with general encoding. Voice segments of power customers whose emotional state meets preset conditions and voice segments related to customer needs are encoded with high fidelity.

[0008] In one implementation of this application, before sending the voice data of the electricity customer to the power service cloud system, the method further includes: capturing the entire continuous digital string of high-frequency electricity words containing numerical entities in the voice data of the electricity customer to obtain key information about the electricity business; wherein, the key information about the electricity business includes one or more of the following: account number, meter number, and fault code; classifying and storing the high-frequency electricity words hierarchically, and storing the local accent expression and Mandarin expression of each high-frequency word accordingly to obtain a hierarchical local accent-Mandarin storage table; wherein, the local accent expression is related to the location of the electricity business hall; the high-frequency electricity categories include: electricity business handling, electricity customer service, electricity price, smart electricity use, and power supply quality and power supply safety; based on the key information about the electricity business, the local accent-Mandarin storage table, and several encoded voice segments of the electricity customer, combined with the context of the electricity customer's speech, completing the fragmented content in the voice data of the electricity customer to obtain the voice data of the electricity customer in the business hall.

[0009] In one implementation of this application, before the power service cloud system extracts features from the voice data of power customers through a pre-defined shared feature extraction network, the method further includes: performing self-supervised pre-training on historical voice data from power marketing service channels to learn context-related feature representations of the historical voice data; wherein, the historical voice data includes inquiries about electricity bill disputes and / or reports of abnormal electricity use and / or inquiries about new business processing; using a multi-head self-attention mechanism to map the context-related feature representations to several subspaces for attention calculation to obtain attention calculation results; wherein, the attention calculation results are used to capture the feature dependencies of power business entities in different subspaces; performing a nonlinear transformation on the attention calculation results through a feedforward neural network to extract high-level abstract features related to power business entities; and performing feature concatenation and dimension mapping between the attention calculation results and the high-level abstract features related to power business entities to construct a shared feature extraction network for power business services.

[0010] In one implementation of this application, the power service cloud system extracts features from the voice data of power customers through a pre-defined shared feature extraction network to determine the customer's emotional state. Specifically, this includes: extracting contextual features from the voice data of power customers through the feature extraction layer of the shared feature extraction network to capture voiceprint features related to the power customer's identity and emotional features related to power business requests; wherein, the emotional features include a hurried tone pattern for electricity bill disputes and / or an anxious tone pattern for electricity repair requests and / or a calm inquiry tone pattern for business consultations; and, based on the multi-head self-attention mechanism module of the shared feature extraction network, performing attention calculation on the voiceprint features and emotional features of power customers, and inputting the attention-calculated voiceprint features and emotional features into a feedforward neural network for nonlinear transformation to obtain the emotional state of the power customer.

[0011] In one implementation of this application, if a customer's emotional state matches a preset special emotional state, one or more video capture devices in the power business hall are controlled to collect video data of the customer's environment. Specifically, this includes: when it is determined that the current customer's emotional state matches a preset special emotional state, querying a list of video capture devices located in the same service area as the intelligent voice recognition device based on the intelligent voice recognition device corresponding to the current customer's emotional state; adjusting the pan-tilt-zoom (PTZ) position and lens field of view of the video capture devices in the list based on the spatial location identifier of the intelligent voice recognition device, so that the center of the captured image is aligned with the service area of ​​the power business hall where the intelligent voice recognition device is located; and transmitting the recorded video data back to the power service cloud system in real time through the internal network of the power business hall.

[0012] In one implementation of this application, before inputting video data, voice data, and customer emotional states into a preset electricity marketing knowledge graph, the method further includes: extracting basic data sources from the electricity marketing business system, performing entity recognition on the basic data sources to extract electricity customer entities, electricity address entities, metering equipment entities, billing item entities, and business work order entities; wherein the basic data sources include customer file data, billing data, electricity metering data, and historical business work order data; establishing semantic relationships between entities based on electricity marketing business procedures; wherein the semantic relationships include the attribution relationship between electricity customers and electricity addresses, the assembly relationship between electricity addresses and metering equipment, the billing relationship between metering equipment and billing items, and the processing relationship between business work orders and electricity customers; performing text mining on historical business work order data to extract fault phenomenon descriptions, customer request statements, and handling measure records from the historical business work order data, and mining causal relationships between fault phenomenon descriptions and handling measure records; combining causal relationships with semantic relationships to form an electricity marketing knowledge graph that includes business logic relationships and fault handling causal relationships.

[0013] In one implementation of this application, the power service cloud system inputs video data, voice data, and customer emotional states into a pre-defined power marketing knowledge graph. Based on the causal relationships within the knowledge graph, it infers the causes of emotions corresponding to the customer's emotional state and the corresponding business needs. Specifically, this includes: extracting behavioral characteristics of the power customer and on-site environmental characteristics from the video data; wherein the behavioral characteristics include at least the range of the power customer's body movements and changes in facial expressions; and the on-site environmental characteristics include at least the number of people queuing around the service station and the relative positions and postures of staff and customers; and concatenating the power customer's emotional state, behavioral characteristics, and on-site environmental characteristics to generate a multi-dimensional... The system retrieves a comprehensive service event description vector and uses it as a query condition to search for power customer entity nodes in the power marketing knowledge graph that match the customer's emotional state. Starting from the power customer entity node, it performs a breadth-first traversal along the preset causal relationship edges in the power marketing knowledge graph, and calculates the similarity between the comprehensive service event description vector and the currently accessed entity attributes and relationship edge weights to filter out candidate entity paths whose relevance to the customer's emotional state meets a preset threshold. The system then identifies the business work order fault type, billing anomaly category, or business processing bottleneck corresponding to the end entity in the candidate entity path as the cause of the emotion and the corresponding business need.

[0014] In one implementation of this application, after determining the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, the method includes: starting from the entity node corresponding to the cause of the emotion, performing a reverse retrieval along the handling measure association edge in the power marketing knowledge graph to obtain the handling measure record corresponding to the cause of the emotion and the corresponding business needs of the customer's emotional state, so as to generate a personalized power marketing response solution containing targeted explanations, specific action suggestions and follow-up service guidance.

[0015] Secondly, this application also provides a response device for electricity business services. The device includes: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: collect voice data of electricity customers in the business hall through an intelligent voice recognition device worn by electricity business hall staff, and send the voice data of electricity customers to an electricity service cloud system; the electricity service cloud system extracts features from the voice data of electricity customers through a preset shared feature extraction network to determine the customer's emotional state; if the customer's emotional state matches a preset special emotional state, it controls one or more video acquisition devices in the electricity business hall to collect video data of the environment in which the electricity customer is located; the electricity service cloud system inputs the video data, voice data, and customer emotional state into a preset electricity marketing knowledge graph, and infers based on the causal relationships in the electricity marketing knowledge graph to determine the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, and displays the corresponding solutions in the form of voice, video, or text.

[0016] The response method and device for electricity business services provided in this application have the following beneficial effects: By collecting voice data in real time through intelligent voice devices, it is possible to identify customers' special emotions such as anxiety, dissatisfaction, and confusion through features such as tone and speed of speech before the customer expresses dissatisfaction or requests for help; when a special emotion is detected, it will link with video acquisition devices to combine voice features, emotion tags, and video images, which can effectively avoid single voice recognition errors caused by accents, noise, recording quality, etc., and form multimodal information complementary verification, which greatly improves the accuracy of judging the customer's real state, on-site environment, and service scenario; the voice data, video data, and customer emotions are input into the electricity marketing knowledge graph, and intelligent reasoning is performed along the causal relationship between business, emotion, and cause in the graph. Based on the reasoning results of the knowledge graph, a standardized solution matching the business type and problem cause is automatically generated, which significantly improves service efficiency and customer experience. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a response method for electricity business services provided in this application embodiment; Figure 2 This is a schematic diagram of the internal structure of a response device for electricity business services provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] This application provides a response method and device for electricity business services to solve the following technical problem: how to trace back from voice, emotion, environment to business root causes.

[0020] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] Figure 1 This is a flowchart illustrating a response method for electricity business services provided in an embodiment of this application. Figure 1 As shown in the figure, the response method for electricity business services provided in this application embodiment specifically includes the following steps: Step 10: Collect voice data of electricity customers in the business hall through the intelligent voice recognition device worn by the staff of the electricity business hall, and send the voice data of electricity customers to the electricity service cloud system.

[0022] As an optional embodiment, the voice data of electricity customers in the business hall is collected by the intelligent voice recognition device worn by the staff of the electricity business hall, and the voice data of the electricity customers is sent to the electricity service cloud system. Specifically, it may include: Step 101: Collect the mixed voice signal of the interaction between the staff of the electricity business hall and the customer in real time by the multi-channel sound pickup device worn by the staff of the electricity business hall.

[0023] In this step, during actual service at power service halls, there are common challenges such as dense crowds in the business processing area, complex background noise, dynamic changes in the distance between customers and staff, and multiple people taking turns consulting. Multi-channel audio pickup devices, deployed in a wearable manner, can move with staff to cover the entire service area, without being limited by fixed locations. They can simultaneously pick up multi-source interactive voice messages, including staff guidance, customer inquiries, questions, and complaint feedback. Through multi-channel signal differentiation and enhancement processing, they effectively suppress environmental interference such as queuing noises, equipment noise, and conversations among surrounding people. Even in high-frequency business scenarios such as customer inquiries about electricity bills and prices, business processing, fault reporting, account cancellation / transfer, and new installations / capacity increases, they can still stably acquire clear and complete interactive voice data. This truly meets the actual needs of mobile service, dynamic interaction, and high-frequency consultation in power service halls, achieving a technological upgrade from passive recording to active, full-scenario, high-fidelity voice acquisition.

[0024] Step 102: Filter out voice segments in the mixed voice signal whose voiceprints are similar to those of the power business hall staff to a preset threshold, identify them as the voices of the power business hall staff, and mark the unmatched voice segments in the mixed voice signal; among them, the unmatched voice segments include customer voices and interference noise.

[0025] In this step, firstly, the voiceprint information of all staff members in the business hall is pre-recorded to establish a dedicated voiceprint database. Considering the professional nature of electricity business services, the voiceprint comparison model will focus on adapting the staff's tone of voice and pace of speech in daily service (such as a gentle pace when guiding customers through business and a clear tone when answering professional questions). When the collected mixed speech signal, after voiceprint feature extraction, reaches a preset threshold in similarity with the staff's voiceprint information in the voiceprint database, that speech segment will be initially identified as a candidate speech segment for the electricity business hall staff. These candidate segments mainly cover various professional expressions used by staff during service, such as explaining the electricity bill payment process to customers, explaining the materials required for new installations and capacity increases, responding to customer questions about electricity pricing policies, and reassuring emotionally agitated customers, ensuring the accuracy and relevance of the speech. The system accurately filters out voice segments from staff members. Simultaneously, it categorizes and labels voice segments in the mixed voice signal that do not reach a preset voiceprint similarity threshold. One category consists of customer voice segments, corresponding to various customer requests and expressions during business transactions, including inquiries about electricity bill details, reporting meter malfunctions, applying for service changes, and complaining about service issues—covering the core customer needs in electricity business services. The other category consists of mismatched interference noise segments, primarily originating from environmental interference in the electricity business hall, such as the calling sound of the queuing machine, the operation prompts of self-service equipment, conversations between other customers and staff, and the operating noise of the business hall's air conditioning and electronic equipment. By clearly labeling customer voice segments and interference noise segments, irrelevant interference information can be effectively removed, preventing noise from interfering with subsequent customer emotion recognition and business intent analysis.

[0026] Step 103: Based on the exclusive interference feature library of the power business hall, filter out unmatched voice segments, remove interference signals in the power business hall scene including self-service payment machine prompts, power terminal buzzing, and queuing machine announcements, and retain candidate power customer voice segments with human voice characteristics.

[0027] In this step, to accurately extract the pure customer voice, a highly scenario-specific interference feature library for power service halls was constructed. This library deeply integrates the unique audio fingerprints of various electronic devices and terminal systems within the service hall. After the mixed voice signal undergoes initial voiceprint screening, a secondary refined filtering process is initiated for voice segments that fail to match the customer's voiceprint. At this point, preset models in the interference feature library are retrieved, and these unmatched segments are compared and matched one by one with the fixed prompts emitted by self-service payment machines during electricity payment operations, the continuous low-frequency humming sound generated by power service terminals during long-term operation, and the standardized synthesized speech of the queuing machine announcing queue numbers. Once a segment is identified as containing the above-mentioned typical interference signals from service hall equipment, it is accurately marked and eliminated. This gradually peels away the audio components that truly contain service value from the complex sound field, namely, customer voice candidate segments with distinct human voice characteristics. The segment selection not only eliminates the interference from the call number machine's announcements, allowing staff to clearly hear customers' faint inquiries in noisy environments, but more importantly, it eliminates the fixed prompts of self-service payment machines and the background buzzing of terminals, thus completely preserving the authentic voice content of customers when inquiring about electricity bills, applying for name changes or transferring ownership, or reporting power outages. For example, when an elderly customer whispers for help in the self-service payment area due to operational errors, by filtering out the frequent prompts of surrounding payment machines, the system can accurately capture the anxious voice segments, providing a clean and high-quality data source for subsequent emotion analysis. This interference filtering mechanism, which is highly aligned with the electricity business scenario, ensures that the intelligent service system can always focus on the actual needs of customers. Whether in the noisy environment during peak business hours or surrounded by various electronic devices, it can accurately pick up and clearly reproduce customer voices, laying a solid data foundation for the intelligent perception of electricity marketing services.

[0028] Step 104: By analyzing the voice timing, business semantic logic, and the voice of the power business hall staff, confirm the correlation between the voice of the power business hall staff and the customer in business interaction. Remove the voices of other irrelevant personnel in the power business hall from the power customer voice candidate segments to obtain the power customer voice signal.

[0029] In this step, from the perspective of voice timing, the business interactions between power service hall staff and customers exhibit clear continuity and correspondence. The staff's service scripts and the customer's voice expressions show an alternating temporal pattern. For example, after the staff explains the electricity bill payment process, the customer immediately raises related questions, or after the staff inquires about the customer's business needs, the customer simultaneously responds to their own requests. Based on this temporal pattern, the temporal correlation between the customer's voice candidate segments and the staff's voice segments is compared, eliminating those voice segments that are not temporally connected to the staff's voice and exist in isolation, such as random conversations that do not correspond to any staff service script or the voices of passersby that are unrelated to the current business processing time. From the perspective of business semantic logic, the interactive content of power service has clear professionalism and relevance, focusing on various power businesses. Combining the pre-set power business semantic library (covering various business-related vocabulary and sentence structures such as electricity bill payment, meter malfunction, business changes, new installations and capacity increases, electricity price consultation, complaints and suggestions, etc.), semantic analysis is performed on the customer's voice candidate segments to determine whether they correspond to the staff's voices during the current business processing time. The content of previous business interactions is highly relevant. For example, when a staff member is processing a customer's electricity bill inquiry, if the customer's voice contains relevant semantics such as "electricity bill details," "payment amount," and "arrears," the relevance can be confirmed. Conversely, if the voice segment contains no electricity business-related semantics, or if the semantic content is unrelated to the business being served by the current staff member (such as business inquiries from customers at adjacent windows or casual conversations by accompanying personnel), it is judged as irrelevant voice and removed. Through dual verification of voice timing and business semantic logic, the system can accurately retain electricity customer voice signals directly related to the staff's business interactions, covering core content such as customer requests, inquiries, and emotional feedback during business processing. At the same time, it can effectively remove various irrelevant voice interferences in the business hall, ensuring that the final electricity customer voice signal is pure and effective, fully adaptable to the needs of subsequent intelligent service links such as customer emotion recognition and business request mining. This aligns with the actual service scenario of multiple staff, multiple windows, and multiple businesses operating in parallel in the electricity business hall, providing high-quality core voice data support for improving the intelligent level of electricity business services.

[0030] Step 105: For multiple voice segments from the same electricity customer, use different encoding modes, and use low bit rate encoding for clear and coherent voice segments.

[0031] In this step, within the service scenario of a power business hall, the same power customer will generate multiple voice segments during the business process. Considering the characteristics of power customer voices, for clear, coherent, and emotionally stable voice segments, such as when customers calmly inquire about business processes or orderly explain their personal needs, a low bitrate encoding mode is adopted. This mode can compress file size to the maximum extent without losing the core semantics of the business, reduce the storage and transmission pressure on the power service cloud system, adapt to the efficient management needs of massive customer voice data in the business hall, and at the same time, it does not affect subsequent business semantic parsing and service traceability, thus fitting the actual scenario of daily power business services.

[0032] Step 106: Use general encoding for non-important voice segments, and use high-fidelity encoding for voice segments where the customer's emotional state meets preset conditions and for voice segments related to the customer's needs.

[0033] In this step, for non-critical voice segments, such as customer casual conversation, repetitive statements, and invalid voice interactions in the business hall environment, a general encoding mode is adopted. This mode ensures basic recognizability while reducing encoding complexity, reducing system storage and transmission pressure, and adapting to the efficient management needs of massive voice data in the business hall. For voice segments where the emotional state of the electricity customer meets preset conditions, such as the hurried voice during electricity bill disputes, the anxious statements when reporting electricity repairs, and voice segments directly related to the customer's core needs, such as explanations of business requests, detailed descriptions of faults, and inquiries about fees, high-fidelity encoding is adopted. This encoding mode can completely preserve the intonation, emotional details, and key points of the request in the voice, ensuring that key voice information is not distorted. This provides accurate support for subsequent customer emotion analysis, business intent interpretation, and problem tracing, and meets the actual needs of intelligent electricity business services.

[0034] Step 107: Capture the high-frequency electricity words containing numerical entities in the voice data of electricity customers as a whole, to obtain key information about electricity business; among which, key information about electricity business includes one or more of the following: account number, meter number, and fault code.

[0035] In this step, when customers inquire about services or report faults, high-frequency numerical entities deeply integrated with electricity services frequently appear, such as ten-digit customer numbers, meter numbers on meter boxes, and fault codes on power outage notices. The customer number is a unique identifier for retrieving customer files, the meter number can locate the physical coordinates of on-site equipment, and the fault code is the technical code for determining the cause of power outages. If any identification is incorrect, it may lead to staff being unable to find customer information or assigning the wrong repair order, causing customers to repeat themselves in frustration and disrupting the service process. By capturing continuous digital strings as a whole, it can maintain high anti-interference capabilities in noisy environments, ensuring the integrity of the digital sequence, thereby quickly triggering the knowledge graph to infer the corresponding power restoration plan or repair instructions, greatly improving the smoothness of business processing in the business hall.

[0036] Step 108: Store high-frequency words related to electricity in a hierarchical and categorized manner, and store the corresponding local accent expressions and Mandarin expressions for each high-frequency word to obtain a hierarchical and categorized local accent-Mandarin storage table; the local accent expressions are related to the location of the electricity business hall; the high-frequency categories of electricity include: electricity business handling, electricity customer service, electricity fees and prices, smart electricity use, and power supply quality and power supply safety.

[0037] In this step, to adapt to regional service needs and improve speech recognition accuracy, high-frequency electricity-related words and phrases need to be stored in a hierarchical and categorized manner, and a correspondence between local accents and standard Mandarin needs to be established. Based on actual business services, high-frequency words and phrases are first divided into five categories, and then managed hierarchically (the hierarchical classification is only an example; users can set the specific level allocation, keyword types, and keywords themselves). Electricity business processing and electricity customer service are set to level A, including new electricity installations, capacity increases / decreases, name changes, etc.; electricity customer service includes business inquiries, fault reporting, and complaint handling; electricity pricing, smart electricity / online services, and power quality and safety are set to level B, including peak-valley pricing and tiered pricing; smart electricity / online services include the State Grid website and smart meters; and power quality and safety includes power reliability and power outages. Simultaneously, local accent expressions are strongly correlated with the location of the business hall; the system will collect commonly used electricity-related words and phrases with accents from customers in the corresponding region and store them in a one-to-one correspondence with standard Mandarin. This system creates a hierarchical and categorized local accent-Mandarin storage table to address recognition errors caused by accent differences among customers from different regions. This helps accurately capture customer business needs and aligns with the actual service scenarios of power service halls in various regions. To improve the accuracy of dialogue understanding, the DialogueBERT model is introduced to perform deep semantic modeling of colloquial expressions. Through role embedding and turn embedding mechanisms, this model can effectively capture the interaction structure between customers and staff in power service dialogues. After the dialect recognition module converts local accents into standard terms, DialogueBERT can accurately understand the core needs of customers by combining the dialogue history context. For example, it can distinguish whether a customer's mention of "tripping" is an inquiry about the cause of the power outage or a report of a fault, thereby accurately triggering subsequent business processes. This fusion mechanism solves the semantic understanding barriers caused by dialects, allowing local customers to experience a more natural and friendly interactive experience when handling business. At the same time, it enhances the inclusiveness and precise reach of power services by leveraging advanced dialogue modeling technology.

[0038] Step 109: Based on key information of the power business, the local accent-Mandarin storage table, and several encoded voice segments of power customers, and combined with the context of the power customers' speech, the fragmented content in the voice data of power customers is supplemented to obtain the voice data of power customers in the business hall.

[0039] In this step, within the intelligent service scenario of the power business hall, customers' spoken expressions are often fragmented and ellipsis-based, frequently omitting only keywords and the complete meaning. To accurately understand customer intent, based on extracted key power business information (such as account number, meter number, and fault code) and a pre-built accent-Mandarin storage table, combined with deep modeling of the dialogue history using the DialogueBERT model, the fragmented content in the voice data is intelligently completed. When a customer says "The meter is off, pay the bill" in dialect, the dialect pronunciation is first converted into standard terminology using the accent-Mandarin storage table, and then combined with... The previously identified account number and service type in the dialogue context are completed as "My electricity meter has stopped, I need to pay to have the power restored"; if the customer only says "The circuit breaker tripped, third floor" in the repair scenario, it is completed as "The circuit breaker tripped in the third floor of my house, please send someone to repair it" based on the address information and fault code recorded in the historical dialogue. This completion mechanism not only transforms the original voice data into semantically complete and logically clear standardized text, providing high-quality input for subsequent knowledge graph reasoning, but also avoids staff having to repeatedly ask for confirmation due to missing information, thereby shortening the business processing time and improving the overall service efficiency and customer satisfaction of the power business hall.

[0040] Step 20: The power service cloud system extracts features from the voice data of power customers through a pre-set shared feature extraction network to determine the customer's emotional state.

[0041] As an optional embodiment, the power service cloud system extracts features from the voice data of power customers through a pre-set shared feature extraction network to determine the customer's emotional state. Specifically, it may include: Step 201: performing self-supervised pre-training on historical voice data from power marketing service channels to learn context-related feature representations of historical voice data; wherein, historical voice data includes inquiries about electricity bill disputes and / or reports of abnormal electricity use and / or inquiries about new business processing.

[0042] In this step, to improve the accuracy of voice recognition and request parsing in electricity business operations, self-supervised pre-training is conducted on historical voice data from electricity marketing service channels. The focus is on learning context-related feature representations of the data, aligning with the core service scenarios of the business hall. The historical voice data originates from daily business services and mainly covers three high-frequency scenarios: first, voice for electricity bill disputes, including customer objections and inquiries regarding electricity bill details, outstanding amounts, and electricity price standards; second, voice for reporting electricity usage anomalies, recording customer requests and descriptions of meter malfunctions, unstable voltage, and power outages; and third, voice for inquiries about new business processing, involving customer consultation scripts for new installations, capacity increases, account transfers, and smart meter applications. Self-supervised pre-training requires no manual annotation and automatically mines the contextual relationships of various voice types, such as the voice association between electricity bill disputes and electricity price policies, and the semantic connection between descriptions of electricity usage anomalies and fault types. It accurately learns the voice features and expression patterns in electricity business scenarios, providing reliable model support for subsequent real-time customer voice emotion recognition and request parsing, adapting to the actual needs of electricity marketing services.

[0043] Step 202: Use a multi-head self-attention mechanism to map the context-related feature representations to several subspaces for attention calculation to obtain attention calculation results; wherein, the attention calculation results are used to capture the feature dependencies of power business entities in different subspaces.

[0044] In this step, to accurately capture the relationships between entities in the power business, a multi-head self-attention mechanism is adopted. The historical speech context-related feature representations obtained through self-supervised pre-training are mapped to several subspaces for attention calculation. This mechanism adapts to the core needs of power marketing services. It can split the attention dimension according to the characteristics of the power business, with different subspaces focusing on the relationship between a type of business entity. For example, the amount of electricity bills and the electricity price standard in electricity bill disputes, the fault phenomenon and the repair type in abnormal electricity use, and the business type and application materials in new business processing. Through parallel calculation in multiple subspaces, the dependencies between different business entities in speech features can be fully explored, and the semantic relationships of the context can be accurately captured. The resulting attention calculation results can clearly show the relationship strength of each business entity, providing accurate support for subsequent analysis of the business intent of customer speech and inference of emotional association.

[0045] Step 203: Perform a nonlinear transformation on the attention calculation results using a feedforward neural network to extract high-level abstract features related to the power business entity.

[0046] In this step, within the intelligent voice processing workflow for electricity sales, the attention calculation results obtained through a multi-head self-attention mechanism are nonlinearly transformed using a feedforward neural network. This process focuses on extracting high-level abstract features related to electricity business entities, aligning with the service scenarios in business halls. This process strips away redundant voice details, focusing on core electricity business entities such as electricity bills, electricity prices, meter malfunctions, and service types. The entity dependencies captured in the attention calculation results are transformed into more representative and business-adaptive high-level features. Combined with high-frequency scenarios in electricity marketing, this process accurately extracts the focus of objections in electricity bill disputes, the core features of faults in abnormal electricity usage, and key information in new business inquiries, eliminating irrelevant interference features. These high-level abstract features more accurately reflect the core business of customer voice, providing efficient support for subsequent customer emotion recognition, business intent judgment, and knowledge graph reasoning. This balances processing efficiency and accuracy, meeting the actual needs of intelligent electricity sales services. Step 204: Perform feature concatenation and dimension mapping between the attention calculation results and the high-level abstract features related to the power business entity to construct a shared feature extraction network for power business services.

[0047] In this step, the attention calculation result obtained through multi-head self-attention mechanism is first concatenated with the high-level abstract features output by the nonlinear transformation of the feedforward neural network. This integrates feature information from different dimensions, allowing the concatenated features to possess both contextual relevance and deep semantic characteristics. Subsequently, a dimension mapping operation is performed on the concatenated features, mapping them to a unified feature dimension space through linear transformations and other methods to eliminate dimensional differences between different features and achieve standardized feature processing. This process closely aligns with the electricity business scenario, focusing on high-frequency scenarios such as electricity bill disputes, electricity anomaly reports, and new business inquiries. Ultimately, a shared feature extraction network for electricity business services is constructed, which can efficiently output speech features that are both relevant and representative.

[0048] Step 205: By using the feature extraction layer of the shared feature extraction network, contextual features are extracted from the voice data of electricity customers to capture voiceprint features related to the identity of electricity customers and emotional features related to electricity business requests. Among them, emotional features include a hurried tone pattern for electricity bill disputes and / or an anxious tone pattern for electricity repairs and / or a calm inquiry tone pattern for business consultations.

[0049] In this step, during the intelligent service process of the power business hall, the shared feature extraction network undertakes the key task of decoding the customer status from the original voice. The network first performs in-depth analysis on the collected customer voice data through the feature extraction layer, and simultaneously captures two types of core features: one is voiceprint features related to customer identity, which is used to confirm the speaker's identity and retrieve the corresponding electricity usage file; the other is emotional features closely related to the electricity business needs. Because customers' voice expression patterns vary significantly across different business scenarios, the feature extraction layer can specifically identify these differentiated tone patterns. For example, in an electricity bill dispute scenario, when a customer questions a sudden increase in electricity costs, their voice often exhibits a rapid, stressed tone. Once the feature extraction layer captures this rapid tone, it can help determine that the customer is in a state of questioning and seeking explanation. In an electricity repair scenario, the anxiety and unease a customer experiences due to a power outage will be reflected in a rising tone and hesitant speech, an anxious tone. This emotional characteristic helps the system prioritize the urgency of the repair request. In daily business consultation scenarios, customers' voices are usually characterized by a steady, calm, and gentle tone. By accurately capturing these scenario-specific emotional features, rich and accurate input data can be provided for subsequent emotional state determination and knowledge graph reasoning.

[0050] Step 206: Based on the multi-head self-attention mechanism module of the shared feature extraction network, attention is calculated on the voiceprint features and emotion features of the electricity customer, and the voiceprint features and emotion features after attention calculation are input into the feedforward neural network for nonlinear transformation to obtain the emotional state of the electricity customer.

[0051] In this step, after the initial extraction of voiceprint and emotion features, these two types of feature vectors are input into the multi-head self-attention mechanism module for deep fusion calculation. This module uses multiple parallel attention heads to calculate the intrinsic correlation weight between voiceprint and emotion features from different dimensions. For example, when the voiceprint features of an elderly customer and the rapid tone of voice are captured at the same time, the attention mechanism will give this combination a higher attention weight to strengthen the recognition of anxious emotions. When the voiceprint of a customer who calls frequently is matched with the calm tone of voice, it may be determined that he is in a routine business consultation state, thus capturing subtle but key emotional cues in complex voice signals.

[0052] Step 30: If the customer's emotional state matches a preset special emotional state, control one or more video acquisition devices in the power business hall to collect video data of the environment where the power customer is located.

[0053] As an optional embodiment, if the customer's emotional state matches a preset special emotional state, one or more video acquisition devices in the power business hall are controlled to collect video data of the environment where the power customer is located. Specifically, this may include: Step 301: When it is determined that the current customer's emotional state matches a preset special emotional state, based on the intelligent voice recognition device corresponding to the current customer's emotional state, a list of video acquisition devices located in the same service area as the intelligent voice recognition device is queried.

[0054] In this step, when it is determined that the current customer's voice characteristics and emotional state match a preset special emotional state, the spatial positioning and device linkage mechanism is immediately triggered. First, the unique identifier of the intelligent voice recognition device currently collecting the customer's voice data is read. This identifier has a fixed mapping relationship with a specific service station or functional area in the business hall, such as the microphone of a certain counter or the pickup device of a certain self-service area. Based on this device identifier, the system automatically queries the pre-configured list of video acquisition device topology relationships in the background and filters out all video acquisition devices in the same physical service area as the voice device, such as the overhead camera of the corresponding counter or the panoramic camera of the service area. This spatial association query ensures that the video resources retrieved subsequently can accurately cover the specific location of the customer, thereby providing real-time video input that matches the environment for multimodal fusion analysis.

[0055] Step 302: Based on the spatial location identifier of the intelligent voice recognition device, adjust the pan-tilt position and lens field of view of the video acquisition device in the video acquisition device list so that the center of the acquisition screen is aligned with the service area of ​​the power business hall where the intelligent voice recognition device is located.

[0056] In this step, since multiple rotatable, zoomable dome cameras are typically deployed in power service halls, and each camera needs to cover multiple service stations, it is necessary to dynamically calculate and adjust the pan-tilt rotation angle and lens focal length of the corresponding camera based on the specific coordinates of the voice equipment. For example, when a customer's emotional fluctuation is detected at counter number 3, based on the fixed spatial location marker of the voice equipment at that counter, a precise pan-tilt control command is sent to the main control camera in that area, causing it to rotate horizontally to the direction of the corresponding counter, adjust the tilt angle to a suitable height for capturing the customer's facial expressions, and automatically zoom in on the lens field of view to capture the interaction between the customer and the staff in front of the counter. The moving area serves as the center of the image; for example, in an electricity repair scenario, when a customer anxiously displays the meter box or uses gestures to describe the fault, the adjusted camera can capture hand movements and meter positions with a suitable focal length; in an electricity bill dispute scenario, the lens can focus on the customer's facial micro-expressions and the staff's explanatory and communication postures; through this dynamic adjustment of the pan-tilt unit and field of view based on spatial location identification, precise spatiotemporal synchronization between the voice source and the video image is achieved, providing high-quality, strongly correlated visual input for subsequent multimodal data fusion analysis, enabling knowledge graph reasoning to provide solutions based on more complete on-site information.

[0057] Step 303: The recorded video data is transmitted back to the power service cloud system in real time through the internal network of the power business hall.

[0058] In this step, after adjusting the pan-tilt-zoom (PTZ) and field of view, and locking onto the customer's service station, the video recording function is automatically activated. This recording is not continuous, but rather triggered on demand when the customer's emotions meet preset special emotional states. This protects customer privacy while ensuring the traceability of key events. Once recording begins, the camera starts capturing high-definition video footage in real time, including the customer's facial expressions, body movements, and the staff's service posture. The generated video data is then transmitted back to the power service cloud system in real time as an encrypted data stream via the high-speed network within the power service hall. Through this on-demand, real-time video stream processing mechanism, the power service hall has built a multimodal perception loop integrating voice, emotion, and visuals, providing solid data support for achieving accurate, efficient, and warm customer service.

[0059] Step 40: The power service cloud system inputs video data, voice data, and customer emotional states into a preset power marketing knowledge graph, and infers based on the causal relationships in the power marketing knowledge graph to determine the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, and displays the corresponding solutions in the form of voice, video or text.

[0060] As an optional embodiment, the power service cloud system receives video data sent by video acquisition equipment, and inputs the video data, voice data, and customer emotional state into a preset power marketing knowledge graph. It then infers along the causal relationships in the power marketing knowledge graph to obtain the business and reasons corresponding to the customer's emotional state, and displays the corresponding solutions in the form of voice, video, or text. Specifically, this may include: Step 401: Extracting basic data sources from the power marketing business system, performing entity recognition on the basic data sources to extract power customer entities, electricity address entities, metering equipment entities, billing item entities, and business work order entities; wherein the basic data sources include customer profile data, billing data, electricity metering data, and historical business work order data.

[0061] In this step, extracting high-quality basic data sources from the power marketing business system is the primary step in constructing the power marketing knowledge graph. These data sources are widely distributed across various business systems, mainly including customer profile data recording customer identity information, billing data storing electricity bill calculation and payment records, electricity metering data collecting meter operating status, and historical business work order data recording past service requests and processing. First, the power customer entity is extracted, including customer name, document type, contact information, and other identity information. Second, the electricity address entity is obtained, clarifying the specific geographical location of the power supply and the corresponding power supply station and line affiliation. Simultaneously, the metering equipment entity is identified, associating the meter number, model, installation date, and current operating status. Furthermore, the billing item entity is extracted, recording electricity price category, tiered usage, historical electricity charges, and outstanding payments. Finally, the business work order entity is obtained, including work order number, business type, acceptance time, and processing result. The complete extraction of these five types of entities provides standardized node definitions and rich attribute information for the subsequent construction of the power marketing knowledge graph.

[0062] Step 402: Based on the electricity marketing business procedures, establish semantic relationships between entities; among which, semantic relationships include the attribution relationship between electricity customers and electricity addresses, the assembly relationship between electricity addresses and metering equipment, the billing relationship between metering equipment and billing items, and the processing relationship between business work orders and electricity customers.

[0063] In this step, during the construction of the electricity marketing knowledge graph, the core entities extracted based on entity recognition, such as electricity customers, electricity addresses, metering equipment, billing items, and business work orders, need to be further established according to the electricity marketing business procedures to establish semantic relationships between them. First, there is the attribution relationship between electricity customers and electricity addresses, clarifying the specific electricity usage location under a particular household name. For example, Mr. Zhang, a residential customer, belongs to a specific apartment in a specific building within a specific community. This relationship is a prerequisite for electricity billing and fault reporting. Second, there is the assembly relationship between electricity addresses and metering equipment, indicating that a meter with a specific number is installed at a specific address, binding the physical space with the specific metering device. Based on this, a billing relationship is established between metering equipment and billing items, meaning that the meter readings are used to generate electricity bills according to the established electricity price policy, linking energy consumption data with economic items. Finally, there is the processing relationship between business work orders and electricity customers, recording when and what kind of business the customer processed, such as name change, capacity increase, or repair report, and returning the processing results to the customer's file.

[0064] Step 403: Perform text mining on historical business work order data to extract fault phenomenon descriptions, customer request statements, and handling measures records from the historical business work order data, and mine causal relationships between fault phenomenon descriptions and handling measures records.

[0065] In this step, the historical work order data contains a wealth of valuable unstructured text information, detailing the entire process of each customer's repair request, inquiry, or complaint. Using natural language processing technology, three key elements are extracted from these historical work orders: First, a description of the fault, such as "the meter's red light is constantly on, and the circuit breaker keeps tripping" or "the electricity bill has doubled compared to last month, I suspect the meter is running fast"; second, the customer's request, such as "request immediate repair," "apply for meter calibration," or "please explain the reason for the abnormal electricity bill"; and finally, a record of the handling measures, i.e., the feedback from staff after on-site handling, such as "inspection revealed that the leakage current protector was aging, and power was successfully restored after replacement" or "after verification..." The platform stated, "Due to the implementation of the third tier of electricity pricing, electricity costs have increased, and this has been explained to customers." Further, association rule mining or deep learning models were used to focus on uncovering stable causal relationships between fault descriptions and handling records. For example, when a large number of work orders simultaneously show fault descriptions of "red light constantly on" and handling records of "battery replacement," causal knowledge of "undervoltage of the meter battery causing the red light to constantly on" can be established. When "tripping" and "replacement of the leakage current device" frequently co-occur, the business logic of "tripping caused by aging leakage current device" can be summarized. These causal relationships extracted from historical practice are solidified into the electricity marketing knowledge graph, becoming the basis for subsequent intelligent reasoning decisions.

[0066] Step 404: Combine causal relationships with semantic relationships to form a power marketing knowledge graph that includes business logic relationships and fault handling causal relationships.

[0067] In this step, semantic relationships form the basic framework of the knowledge graph, clarifying the standard business links between electricity customers, electricity addresses, metering equipment, billing items, and business work orders. For example, the customer's home address, the electricity meter installed at the address, and the billing item associated with the meter. Causal relationships record the inherent logic between fault phenomena and handling measures. By combining the two, when a customer questions a sudden increase in electricity bills, the semantic relationships can be used to locate the specific customer's billing item and electricity consumption curve. Then, the causal relationships can be used to determine whether it is caused by seasonal tiered electricity price changes or requires triggering a check for abnormal metering equipment. When a customer reports a power outage, the metering equipment is located starting from the electricity address and then the causal relationships are used to quickly match the handling measures corresponding to similar fault phenomena in historical work orders. This knowledge graph, which integrates business logic relationships and fault handling causality, enables the intelligent service system of the power business hall to have end-to-end reasoning capabilities from customer voice and emotions to business processing and fault resolution, realizing a truly knowledge-driven proactive service.

[0068] Step 405: Extract the behavioral characteristics of the electricity customer and the characteristics of the on-site environment from the video data; wherein, the behavioral characteristics include at least the range of the electricity customer's body movements and changes in facial expressions; the on-site environmental characteristics include at least the number of people queuing around the service station and the relative positions and postures of the staff and customers.

[0069] In this step, after the video capture device accurately focuses on the target service station and transmits the image, the video stream is analyzed frame by frame to capture the behavioral characteristics of electricity customers and the characteristics of the on-site environment. Special attention is paid to the range of the customer's body movements and changes in facial expressions. For example, when a customer is emotionally agitated due to a dispute over electricity bills, their movements may increase significantly, exhibiting frequent waving or leaning forward. Facial expression analysis can identify subtle changes such as frowning and drooping corners of the mouth, which can be used to verify the judgment of emotional characteristics in the voice. Regarding the extraction of on-site environmental features, the number of people waiting in line around the service station is considered to assess the impact of the current level of congestion in the business hall on customer emotions. In addition, the relative positions and postures of staff and customers are also included in the analysis. For example, whether the staff is standing patiently explaining or sitting while handling documents, and whether the distance between them is appropriate, these details directly reflect the quality of interaction in the service process. Through the dual extraction of behavioral and environmental features, a three-dimensional on-site perception view can be constructed, providing a comprehensive decision-making basis for subsequent causal reasoning.

[0070] Step 406: Combine the emotional state, behavioral characteristics, and on-site environmental characteristics of the electricity customer to generate a comprehensive service event description vector containing multi-dimensional information. Then, use the comprehensive service event description vector as the query condition to retrieve the electricity customer entity node in the electricity marketing knowledge graph that matches the customer's emotional state.

[0071] In this step, the customer emotional state output by the voice analysis module, the behavioral features extracted by the video analysis module, and the on-site environmental features are first vectorized. These heterogeneous information elements are then integrated into a unified, semantically rich, comprehensive service event description vector through a concatenation operation. This vector not only contains customer emotional information but also integrates customer behavior and contextual information, forming a three-dimensional, multi-dimensional digital mirror of the current service event. Subsequently, using this comprehensive service event description vector as query conditions, semantic retrieval and similarity matching are performed within the constructed power marketing knowledge graph. The power customer entity nodes in the knowledge graph are pre-associated with... Rich attribute information, including historical service records, past emotional state tags, and electricity consumption behavior patterns, can quickly locate the electricity customer entity node most similar to the current comprehensive service event description through distance calculation in vector space. For example, when the comprehensive vector shows a combination of features such as "large gestures + frowning + long queues + anxious emotions," it can retrieve customer entities that have historically raised concerns about long wait times during peak hours and access their corresponding handling experience and solutions. The entity retrieval mechanism based on multimodal fusion vectors can understand the current customer from the perspective of historical experience, providing a precise starting point for subsequent causal reasoning and personalized service responses.

[0072] Step 407: Starting from the electricity customer entity node, perform a breadth-first traversal along the preset causal relationship edges in the electricity marketing knowledge graph, and calculate the similarity between the comprehensive service event description vector and the currently accessed entity attributes and relationship edge weights to filter out candidate entity paths that meet the preset threshold for correlation with the customer's emotional state.

[0073] In this step, starting with the retrieved electricity customer entity node, a causal traversal based on a breadth-first strategy is initiated. This process unfolds layer by layer along the pre-defined causal relationship edges in the knowledge graph. For example, starting from the customer entity, it sequentially visits its associated electricity address, the metering equipment installed at that address, the billing items associated with the equipment, and historical business work order entities related to the customer. In each layer of traversal, the similarity calculation is performed in real time between the previously generated integrated service event description vector and the attributes of the currently visited entity (such as meter fault codes, historical work order processing results) and the weights of the relationship edges to quantitatively assess the degree of matching between the current path and the customer's emotional state; for example, when the customer shows signs of anxiously reporting a repair... In the emotional state, starting from the customer entity, the process traverses through the electricity address to the metering equipment entity. If a recent fault record of the equipment with a constantly lit red light is found, and the behavioral characteristics in the comprehensive service event vector include the action of pointing at the meter box, the similarity score of the current path will increase significantly. Candidate entity paths with similarity that meet the preset threshold are continuously filtered out, and branches with low relevance to the current emotional state, such as routine consultation records unrelated to billing disputes, are eliminated. Through this breadth-first traversal with layer-by-layer filtering, one or more causal paths can be accurately identified, starting from the current customer, passing through relevant business entities, and ultimately pointing to potential causes and solutions. This provides a structured reasoning basis for subsequent output of targeted handling suggestions.

[0074] Step 408: Determine the business work order fault type, billing anomaly category, or business processing bottleneck corresponding to the end entity in the candidate entity path as the cause of the emotion and the corresponding business needs corresponding to the customer's emotional state.

[0075] In this step, through layer-by-layer screening and deep tracking of candidate entity paths, the final entity node at the end of the path is identified, and its associated business information is determined as the cause of the customer's current emotional state. This terminal entity may be a fault type recorded in historical business orders, such as a path pointing to specific fault records like low battery voltage in the meter or aging leakage protection device; it may also be an abnormal category marked in the billing account entity, such as accounting factors leading to a sudden increase in electricity bills, such as tiered electricity pricing jumps, historical estimated meter readings being recalculated, or late payment penalties being incorrectly calculated; or it may be a processing bottleneck exposed in the business order flow, such as repeated order cancellations due to incomplete information, time-out process approvals, or failure to arrange on-site inspections on schedule. By identifying this terminal entity... These business tags are mapped to the customer's emotional state as determined in the early stages, allowing for explanation from the root cause of the business. For example, when a customer's emotional state is questioning and urgent, if the path ends at the third tier of the tiered electricity price in the billing item, the cause of the emotion can be determined to be a seasonal increase in electricity consumption leading to a price jump. If the customer exhibits panic and anxiety and the path ends at the flashing yellow light on the meter in the work order, the cause can be determined to be a meter malfunction causing a panic about power outages. This attribution mechanism based on graph traversal can penetrate the surface of customer emotions and reach the internal logic of the power business, providing a solid causal basis for subsequent accurate solution delivery, thereby achieving an intelligent leap from perceiving emotions to understanding the root cause in the business hall scenario.

[0076] Step 409: Starting from the entity node corresponding to the cause of the emotion, perform a reverse retrieval along the handling measures association edge in the power marketing knowledge graph to obtain the records of the cause of the emotion corresponding to the customer's emotional state and the corresponding handling measures corresponding to the business needs, so as to generate a personalized power marketing response solution that includes targeted explanations, specific action suggestions and follow-up service guidance.

[0077] In this step, starting with the identified cause of the emotion, a reverse retrieval is performed along the pre-defined handling measures association edges in the knowledge graph. These association edges connect various business cause entities with their corresponding historical handling measure records. For example, the low battery voltage node is associated with the operation instructions for replacing and reactivating the battery; the tiered electricity pricing jump node is associated with the explanatory script for providing peak-valley time-of-use pricing optimization suggestions; and the incomplete data cancellation node is associated with the service process of providing a one-time supplementary document list and online pre-review channel. Through this reverse retrieval, standardized handling solutions that have been proven in practice can be quickly retrieved from the graph. After obtaining the handling measure records, the cloud platform combines the current customer's voice emotion, behavioral characteristics, and environmental factors to transform these basic measure records into personalized marketing service response strategies. For example, when the potential cause is an aging residual current device (RCD) and the customer is anxious, the generated strategy includes a targeted explanation, such as "Your circuit breaker tripped due to an aging RCD. We have notified repair personnel to arrive within half an hour with new parts," as well as follow-up service guidance, such as "Please observe the usage of high-power appliances such as air conditioners after replacement." If the cause is a "misunderstanding of tiered pricing" and the customer's tone is calm, the strategy focuses on explaining the pricing standards for each tier in detail and proactively pushes the entry point for the electricity usage analysis function of the "Mobile Power APP." This response strategy generation mechanism based on knowledge graph reverse retrieval ensures that every explanation and every suggestion originates from real business logic and historical experience, achieving a deep integration of emotional perception and business handling, making the service response of the power business hall not only fast but also accurate and warm.

[0078] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a response device for electricity business services, the structure of which is as follows: Figure 2 As shown.

[0079] Figure 2 This is a schematic diagram of the internal structure of a response device for electricity business services, provided as an embodiment of this application. Figure 2 As shown, the device includes: At least one processor 201; And a memory 202 that is communicatively connected to at least one processor; The memory 202 stores instructions executable by at least one processor. These instructions are executed by at least one processor 201 to enable the processor 201 to: collect voice data of electricity customers in the electricity business hall through a smart voice recognition device worn by staff, and send the voice data to the electricity service cloud system; the electricity service cloud system extracts features from the voice data of electricity customers through a preset shared feature extraction network to determine the customer's emotional state; if the customer's emotional state matches a preset special emotional state, the system controls one or more video acquisition devices in the electricity business hall to collect video data of the environment in which the electricity customer is located; the electricity service cloud system inputs the video data, voice data, and customer emotional state into a preset electricity marketing knowledge graph, and infers based on the causal relationships in the electricity marketing knowledge graph to determine the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, and displays the corresponding solutions in the form of voice, video, or text.

[0080] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A response method for electricity business services, characterized in that, The method includes: The system collects voice data from electricity customers in the service hall using intelligent voice recognition devices worn by staff, and then sends the voice data to the electricity service cloud system. The power service cloud system extracts features from the voice data of the power customer through a preset shared feature extraction network to determine the customer's emotional state. If the customer's emotional state matches a preset special emotional state, control one or more video acquisition devices in the power business hall to collect video data of the environment where the power customer is located. The power service cloud system inputs the video data, voice data, and customer emotional state into a preset power marketing knowledge graph, and performs reasoning based on the causal relationships in the power marketing knowledge graph to determine the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, and displays the corresponding solutions in the form of voice, video, or text.

2. The response method for electricity business services according to claim 1, characterized in that, Voice data from electricity customers is collected through smart voice recognition devices worn by staff at electricity service halls. Specifically, this includes: The system uses multi-channel audio pickup devices worn by power business hall staff to collect mixed voice signals from interactions between power business hall staff and customers in real time. Voice segments in the mixed voice signal whose voiceprints have a similarity to the voiceprints of power business hall staff members reach a preset threshold are selected, identified as the voices of power business hall staff members, and unmatched voice segments are marked in the mixed voice signal; wherein, the unmatched voice segments include customer voices and interference noise; Based on the exclusive interference feature database of the power business hall, the unmatched voice segments are filtered out, and interference signals in the power business hall scene, including self-service payment machine prompts, power terminal buzzing, and queuing machine announcements, are removed, while candidate voice segments of power customers with human voice characteristics are retained. By analyzing voice timing, business semantic logic, and the voice of the staff at the power business hall, the correlation between the voice of the staff at the power business hall and the voice of the customer is confirmed. The voices of other irrelevant personnel in the power business hall are removed from the candidate voice segments of the power customer to obtain the voice signal of the power customer. For multiple voice segments from the same electricity customer, different encoding modes are used. Low bit rate encoding is used for clear and coherent voice segments; general encoding is used for unimportant voice segments; and high-fidelity encoding is used for voice segments where the electricity customer's emotional state meets preset conditions and voice segments related to the customer's needs.

3. The response method for electricity business services according to claim 2, characterized in that, Before sending the voice data of the electricity customer to the electricity service cloud system, the method further includes: The voice data of the electricity customer containing high-frequency electricity-related words with numerical entities is captured as a whole by continuous digital string capture to obtain key information about the electricity business; wherein, the key information about the electricity business includes one or more of the following: account number, meter number, and fault code; High-frequency words related to electricity are hierarchically categorized and stored, and the local accent expressions of each high-frequency word are stored in correspondence with their Mandarin expressions to obtain a hierarchical and categorized local accent-Mandarin storage table; wherein, the local accent expressions are related to the location of the electricity business hall; the high-frequency categories of electricity include: electricity business handling, electricity customer service, electricity fees and prices, smart electricity use, and power supply quality and power supply safety; Based on the key information of the power business, the local accent-Mandarin storage table, and several encoded voice segments of power customers, and combined with the context of the power customers' speech, the fragmented content in the voice data of the power customers is supplemented to obtain the voice data of the power customers in the business hall.

4. The response method for electricity business services according to claim 1, characterized in that, Before the power service cloud system extracts features from the voice data of the power customer through a preset shared feature extraction network, the method further includes: Self-supervised pre-training is performed on historical voice data from electricity marketing service channels to learn context-related feature representations of the historical voice data; wherein, the historical voice data includes inquiries about electricity bill disputes and / or reports of abnormal electricity use and / or inquiries about new business processing; A multi-head self-attention mechanism is used to map the context-related feature representation to several subspaces for attention calculation to obtain attention calculation results; wherein, the attention calculation results are used to capture the feature dependencies of power business entities in different subspaces; The attention calculation results are nonlinearly transformed by a feedforward neural network to extract high-level abstract features related to the power business entity. The attention calculation results are combined with the high-level abstract features related to the power business entity through feature concatenation and dimension mapping to construct a shared feature extraction network for power business services.

5. A response method for electricity business services according to claim 4, characterized in that, The power service cloud system extracts features from the voice data of the power customers through a pre-set shared feature extraction network to determine the customer's emotional state, specifically including: Through the feature extraction layer of the shared feature extraction network, contextual features are extracted from the voice data of the electricity customer to capture voiceprint features related to the electricity customer's identity and emotional features related to electricity business requests in the voice data; wherein, the emotional features include a hurried tone pattern for electricity bill disputes and / or an anxious tone pattern for electricity repairs and / or a calm inquiry tone pattern for business consultations. Based on the multi-head self-attention mechanism module of the shared feature extraction network, attention is calculated on the voiceprint features and emotional features of the power customer, and the voiceprint features and emotional features after attention calculation are input into the feedforward neural network for nonlinear transformation to obtain the emotional state of the power customer.

6. The response method for electricity business services according to claim 1, characterized in that, If the customer's emotional state matches a preset specific emotional state, one or more video acquisition devices in the power service hall are controlled to collect video data of the customer's environment, specifically including: When it is determined that the current customer's emotional state matches the preset special emotional state, based on the intelligent voice recognition device corresponding to the current customer's emotional state, the list of video capture devices that are in the same service area as the intelligent voice recognition device is queried. Based on the spatial location identifier of the intelligent voice recognition device, adjust the pan-tilt position and lens field of view of the video acquisition devices in the video acquisition device list so that the center of the acquired image is aligned with the service area of ​​the power business hall where the intelligent voice recognition device is located. The recorded video data is transmitted back to the power service cloud system in real time via the internal network of the power business hall.

7. The response method for electricity business services according to claim 1, characterized in that, Before inputting the video data, voice data, and customer emotional state into a preset electricity marketing knowledge graph, the method further includes: The basic data source is extracted from the power marketing business system, and entity identification is performed on the basic data source to extract the power customer entity, electricity address entity, metering equipment entity, billing item entity, and business work order entity; wherein the basic data source includes customer profile data, billing data, electricity metering data, and historical business work order data. Based on the electricity marketing business procedures, semantic relationships between entities are established; wherein, the semantic relationships include the attribution relationship between electricity customers and electricity addresses, the assembly relationship between electricity addresses and metering equipment, the billing relationship between metering equipment and billing items, and the processing relationship between business work orders and electricity customers; Text mining is performed on the historical business work order data to extract fault phenomenon descriptions, customer request statements and handling measures records from the historical business work order data, and causal relationships are mined from the fault phenomenon descriptions and handling measures records. The causal relationships are combined with the semantic relationships to form a power marketing knowledge graph that includes business logic relationships and fault handling causal relationships.

8. A response method for electricity business services according to claim 7, characterized in that, The power service cloud system inputs the video data, voice data, and customer emotional state into a preset power marketing knowledge graph, and infers based on the causal relationships in the power marketing knowledge graph to determine the causes of the emotions corresponding to the customer's emotional state and the corresponding business needs, specifically including: The behavioral characteristics of the electricity customer and the characteristics of the on-site environment are extracted from the video data; wherein, the behavioral characteristics include at least the range of the electricity customer's body movements and changes in facial expressions; the on-site environmental characteristics include at least the number of people queuing around the service station and the relative positions and postures of the staff and customers; The emotional state of the electricity customer, the behavioral characteristics, and the on-site environmental characteristics are concatenated to generate a comprehensive service event description vector containing multi-dimensional information. The comprehensive service event description vector is then used as a query condition to retrieve electricity customer entity nodes in the electricity marketing knowledge graph that match the customer's emotional state. Starting from the electricity customer entity node, a breadth-first traversal is performed along the preset causal relationship edges in the electricity marketing knowledge graph, and the similarity between the comprehensive service event description vector and the currently accessed entity attributes and relationship edge weights is calculated to filter out candidate entity paths that meet the preset threshold in terms of correlation with the customer's emotional state. The business work order fault type, billing anomaly category, or business processing bottleneck corresponding to the end entity in the candidate entity path is determined as the cause of the emotion and the corresponding business need corresponding to the customer's emotional state.

9. A response method for electricity business services according to claim 8, characterized in that, After determining the cause of the emotion corresponding to the customer's emotional state and the corresponding business needs, the method includes: Starting from the entity node corresponding to the cause of the emotion, a reverse retrieval is performed along the handling measures association edge in the power marketing knowledge graph to obtain the records of the cause of the emotion corresponding to the customer's emotional state and the corresponding handling measures corresponding to the business needs, so as to generate a personalized power marketing response solution that includes targeted explanations, specific action suggestions and follow-up service guidance.

10. A response device for electricity business services, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a method as described in any one of claims 1-9.

Images