Device and method for intelligently supervising on-site communication of power utilization appeals between power supply service personnel and user

Abstract

The invention relates to a device and method for intelligently supervising on-site communication of power utilization appeals between power supply service personnel and users, and belongs to the technical field of power supply service supervision. According to the technical scheme, a system for intelligently supervising the reply content and mode when a power supply service worker replies the power utilization appeal proposed by a user on a power supply service site is constructed based on a work recorder; the system comprises a power utilization appeal question and answer skill library, a field personnel identity recognition module, a voice-to-text module, a text-to-feature vector module, a language dialogue matching module and an early warning module, and real-time supervision of communication contents between service personnel and users in a power supply service field is realized through a work recorder and an intelligent algorithm model. According to the invention, accurate capture of user appeals in the whole process of the service site can be realized, so that a traditional language dialogue supervision mode of the work site of power supply service personnel is replaced, the supervision process is objective and intelligent, the supervision method is flexible, and waste of resources is reduced.

Description

Technical Field

[0001] The present invention relates to a device and method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, belonging to the technical field of power supply service supervision. Background Art

[0002] During the power supply service process, effective communication between power supply service personnel and users is a key link in shaping the service image of the State Grid Corporation. At the actual service site, users often raise various issues and demands regarding electricity use to power supply service personnel. If power supply service personnel fail to respond to these demands in a timely or comprehensive manner, it may lead to user misunderstandings, which in turn greatly increases the company's service risks. The failure of power supply service personnel to properly respond to users' electricity demands at the service site is one of the important reasons for user dissatisfaction and reduced service perception, and even user complaints.

[0003] At present, in order to standardize the content and method of power supply service personnel's responses to users' electricity consumption demands on site and avoid a decline in users' service perception, State Grid Corporation of China usually requires power supply service personnel to supervise by wearing traditional work recorders. Traditional work recorders only have functions such as recording. The supervision method is usually as follows: power supply service personnel wear work recorders → turn on work recorders to record the site → end recording of the work site → save the video to a storage device → find the saved video for investigation and evidence collection if evidence needs to be verified later; such recorders can collect voice information at the service site to provide supporting evidence for subsequent problem searches that may be needed. The method of wearing traditional work recorders to supervise the power supply service site is mainly used for record retention. This supervision method mainly relies on the self-discipline of power supply service personnel and does not have the function of supervising the real-time service process.

[0004] In addition, there are also systems and methods for supervising the service behavior of power supply service personnel based on the control background, which mainly monitors the work behavior of power supply service personnel in fixed work scenes in real time through monitoring devices. Supervision method: The monitoring device obtains the work behavior of power supply service personnel in real time → The supervisor retrieves and checks the work site through the supervision background → Manually judges in real time whether there are any violations by the on-site power supply service personnel → Warning notification for violations. The system that supervises the content of the power supply service personnel's on-site power demand response based on the control background requires manual real-time supervision of the power supply service site. There are multiple problems such as manual supervision is time-consuming and labor-intensive, the system development cost is high, and the supervision process requires the participation of multiple links; Due to the diversity of power supply service scenarios, the efficiency of manual supervision is difficult to reach 100%, and it is impossible to achieve the effect of full coverage of all sites and full real-time supervision and warning. In addition, manual supervision is subjective and cannot guarantee the objectivity of the on-site power supply service situation supervision.

[0005] Therefore, as the difficulty of service increases, there is an urgent need to find a method that can effectively monitor the content and manner in which power supply service personnel respond to users' electricity demands in real time throughout the service site, so as to completely eliminate user prejudice caused by low communication quality and inappropriate response to demands, and improve the company's service quality and service image. Summary of the invention

[0006] The purpose of the present invention is to provide a device and method for intelligently supervising power supply service personnel to communicate with users on-site about their power consumption demands, which can achieve accurate capture of user demands throughout the service site, thereby replacing the traditional language dialogue supervision method of power supply service personnel at the work site. The supervision process is objective and intelligent, and the supervision method is flexible. Supervision using this method and device does not affect on-site work at all, and innovates the supervision method, reduces waste of resources, and solves the above-mentioned technical problems existing in the existing technology.

[0007] The technical solution of the present invention is:

[0008] A device for intelligently supervising power supply service personnel and users' on-site communication on power demand, which is used in the scenario where the staff of the State Grid power supply service site respond to the power demand of users on site. A system is constructed based on a work recorder to intelligently supervise the content and method of the response of power supply service personnel to the power demand raised by users at the power supply service site. The system includes a power demand question and answer script library, an on-site personnel identity recognition module, a speech-to-text module, a text-to-feature vector module, a language dialogue matching module and an early warning module. The real-time supervision of the communication content between the service personnel and users at the power supply service site is realized through the work recorder and the intelligent algorithm model; the power supply service personnel and users are automatically identified through the on-site personnel identity recognition module, and the speech of the on-site personnel dialogue content is automatically converted through the speech-to-text module. The data is converted into text, and the feature vector is found through the text-to-feature vector module. The language dialogue matching module is used to perform similarity matching with the feature vector in the electricity demand question and answer script library. The electricity demand and the corresponding demand reply script that best match the on-site dialogue content are found in the electricity demand question and answer script library. By comparison, it is automatically identified whether there are electricity-related demands in the content of communication between the user and the power supply service personnel during the power supply service process. Once it is detected that the user in the conversation has raised an electricity demand, the corresponding reply content of the power supply service personnel will be supervised and judged according to the service specifications of the State Grid Corporation. If it is found that the reply content of the power supply service personnel is wrong or incomplete, the early warning mechanism will be immediately triggered through the early warning module, and the power supply service personnel will be immediately reminded to standardize their service script to ensure service quality.

[0009] The work recorder may also be other portable devices, which need to include functions such as a camera and a recording device.

[0010] A method for intelligently supervising power supply service personnel in on-site communication with users on power demand, based on a work recorder, constructs a system for intelligently supervising the content and method of the response of power supply service personnel to the power demand raised by the user at the power supply service site, automatically identifies the power supply service personnel and the user through the on-site personnel identity recognition module, automatically converts the speech of the on-site personnel dialogue content into text through the speech-to-text module, finds the feature vector through the text-to-feature vector module, performs similarity matching with the feature vector in the power demand question-and-answer script library through the language dialogue matching module, finds the power demand and the corresponding demand response script that best match the on-site dialogue content and the on-site dialogue content in the power demand question-and-answer script library, and automatically identifies whether there is a demand related to electricity in the content of the communication between the user and the power supply service personnel during the power supply service process through comparison; once it is detected that the user in the dialogue has raised an electricity demand, the corresponding response content of the power supply service personnel will be supervised and judged according to the service specifications of the State Grid Corporation; if it is found that the response content of the power supply service personnel is wrong or the response content is incomplete, the early warning mechanism will be immediately triggered through the early warning module, and the power supply service personnel will be immediately reminded to standardize their service scripts to ensure service quality.

[0011] The method specifically comprises the following steps:

[0012] Step 1: Establish a Q&A database for electricity demand;

[0013] According to the power supply service specifications of the State Grid and the historical user demands received by the State Grid 95598 hotline, a Q&A script library for power supply service questions and answers is established; the content of the Q&A script library for power demand is in the form of one question and one answer, and the relevant content is the power demand and the response to the demand; a text library of user power demand and a text library of corresponding demand responses of power supply service personnel are established respectively;

[0014] Step 2: Establish an on-site personnel identification module;

[0015] First, the on-site voice is collected through the work recorder; then, the voice of the on-site power supply service personnel and other voices are identified using voiceprint biometrics technology; after that, the identity of the person in the conversation is determined in real time; that is, if the voice of the power supply service personnel is identified, the speaker of the voice is determined to be the power supply service personnel, and if the voice of the power supply service personnel is not identified, the speaker of the voice is determined to be the user;

[0016] Step 3: Create a speech-to-text module;

[0017] Establish a voice-to-text module for power supply service personnel and users at the service site; use the power supply service personnel's voice and user's voice obtained in step 2 to carry out voice-to-text conversion for the on-site power supply service personnel and users based on the Whisper model. By establishing a voice-to-text module, the voices of users and on-site service personnel are converted into user text A and power supply service personnel text B at the service site for subsequent text library matching;

[0018] Step 4: Establish a text-to-feature vector module;

[0019] Through the user text A and the power supply service personnel text B output in step 3, the important feature vectors in the two texts are found respectively; by establishing a text-to-feature vector module, the user text A and the power supply service personnel text B are converted into feature vectors at the service site for subsequent dialogue question-answer matching;

[0020] Step 5: Establish a language dialogue matching module;

[0021] The feature vector of the power supply service personnel text B and the feature vector of the user text A extracted in step 4 are matched with the feature vectors of the user power demand text library and the demand answer text library in the power demand question and answer script library respectively; after feature vector matching similarity matching, the power demand and the corresponding demand answer script that best match the on-site conversation content are found in the power demand question and answer script library;

[0022] Feature vector matching similarity refers to measuring the similarity between two or more feature vectors by calculating the distance or similarity between them in fields such as computer vision or natural language processing;

[0023] Step 6: Establish an early warning module;

[0024] When it is determined through step 5 that the power supply service personnel have not properly responded to the user's service request, a warning signal is quickly issued to the power supply service personnel, requiring them to carefully and comprehensively answer the user's related demand questions to avoid inappropriate handling of user requests.

[0025] The power supply service site refers to the scene where staff provide electricity services to users, and the scope is not limited to power supply business halls, power consumption sites, etc.

[0026] The 95598 hotline in step 1 refers to the customer service hotline of State Grid Corporation of China.

[0027] The voiceprint biometrics in step 2 is a technology that verifies identity by analyzing personal voice characteristics. The process of establishing voiceprint recognition for power supply service personnel is divided into two stages: voiceprint modeling for power supply service personnel and voiceprint recognition for power supply service personnel:

[0028] 1) Voiceprint modeling of power supply service personnel

[0029] The power supply service personnel read a set of specific sentences or digital strings according to the instructions; then, the collected audio data is pre-processed using an audio processing device to ensure the audio quality; secondly, the acoustic characteristics of the power supply service personnel are extracted; finally, a voiceprint template of the power supply service personnel is established;

[0030] 2) Voiceprint recognition for power supply service personnel

[0031] By verifying the voiceprint of the power supply service personnel and preprocessing the on-site audio data; then, using the on-site audio data to extract acoustic features; after that, comparing the newly extracted features with the established voiceprint template; finally, if the degree of match is higher than the predetermined threshold, the person is verified as a power supply service personnel; otherwise, the person is verified as a customer; by identifying the voiceprint of the power supply service personnel in the on-site voice, the identity of the person during the conversation is determined to prepare for subsequent work.

[0032] The Whisper model of step 3 has an architecture based on Transformer, and its input is a logarithmic Mel-spectrogram of 80 channels; the encoder of the Whisper model consists of two convolutional layers with a convolution kernel size of 3, sinusoidal position encoding, and a series of stacked transformer blocks; the decoder also uses the learned position embedding and uses the same number of transformer blocks as the encoder; the model adopts a multi-task training format, and performs various speech processing tasks through a single decoder training model; multi-task training is achieved by conditioning the decoder on an input token sequence of a specified task and expected output format; the speech of the on-site power supply service personnel and the user is processed into a logarithmic Mel-spectrogram respectively, and used as input using the whisper model to output the power supply service personnel's speech text and the user's speech text.

[0033] The text-to-feature-vector related model algorithm of the text-to-feature-vector module in step 4 adopts the BERT model. The BERT model is a pre-trained language model based on the Transformer architecture. The performance of the BERT model in multiple natural language processing tasks is significantly improved, especially in question answering, text classification, named entity recognition, etc. The BERT model uses the encoder-decoder part of a multi-layer Transformer as its main architecture.

[0034] The present invention uses a work recorder combined with an intelligent model for supervision, strengthens the supervision of the service dialogue of power supply service personnel, and improves the user's service perception. When the power supply service personnel do not respond properly to the user's service request on site, the service personnel can be immediately warned and required to respond accurately to the user's request. The language content of the power supply service personnel can also be monitored in real time, effectively avoiding the problem of poor user service perception due to unclear responses from service personnel.

[0035] The beneficial effects of the present invention are: through the work recorder and the intelligent algorithm model, real-time supervision of the communication content between the power supply service personnel and the user at the power supply service site can be achieved, and whether there are demands related to electricity consumption in the content of the communication between the user and the power supply service personnel during the power supply service process can be automatically identified. Once it is detected that the user in the conversation has raised a demand for electricity consumption, the corresponding reply content of the power supply service personnel will be strictly supervised and judged in accordance with the service specifications of the State Grid Corporation. If it is found that the reply content of the power supply service personnel is wrong or the reply content is incomplete, the early warning mechanism will be triggered immediately, and the power supply service personnel will be reminded immediately, requiring them to standardize their service words to ensure service quality. The present invention can realize the accurate capture of user demands throughout the service site, thereby replacing the traditional language dialogue supervision method of the power supply service personnel at the work site. The supervision process is objective and intelligent, and the supervision method is flexible. The use of this method and device for supervision does not affect the on-site work at all, and it innovates the supervision method and reduces the waste of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the module structure of an embodiment of the present invention;

[0038] Figure 3 This is a flow chart of on-site personnel identification according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the speech-to-text model architecture of an embodiment of the present invention;

[0040] Figure 5 The following is a flow chart of speech-to-text conversion based on the Whisper model according to an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the text feature extraction model architecture of an embodiment of the present invention;

[0042] Figure 7 This is a flowchart of converting text to feature vectors based on the BERT model in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The invention is further described below by way of examples with reference to the accompanying drawings.

[0044] Refer to the attached Figure 1 , 2, a device for intelligently supervising power supply service personnel to communicate with users on site about power demand, which is used in the scenario where the staff of the State Grid power supply service site respond to the power demand of users on site. A system is constructed based on a work recorder to intelligently supervise the content and method of the response of power supply service personnel to the power demand raised by users on site. The system includes a power demand question and answer script library, a site personnel identity recognition module, a speech-to-text module, a text-to-feature vector module, a language dialogue matching module and an early warning module. The real-time supervision of the communication content between the service personnel and users on site of power supply service is realized through the work recorder and the intelligent algorithm model; the power supply service personnel and users are automatically identified through the site personnel identity recognition module, and the speech of the on-site personnel dialogue content is automatically converted through the speech-to-text module. The system automatically converts the speech into text, finds the feature vector through the text-to-feature vector module, performs similarity matching with the feature vector in the electricity demand question and answer script library through the language dialogue matching module, finds the electricity demand and the corresponding demand reply script that best match the on-site dialogue content in the electricity demand question and answer script library, and automatically identifies whether there are electricity-related demands in the content of communication between users and power supply service personnel during the power supply service process through comparison; once it is detected that the user in the conversation has raised an electricity demand, the corresponding reply content of the power supply service personnel will be supervised and judged according to the service specifications of the State Grid Corporation; if it is found that the reply content of the power supply service personnel is wrong or incomplete, the early warning mechanism will be immediately triggered through the early warning module, and the power supply service personnel will be immediately reminded to standardize their service script to ensure service quality.

[0045] The work recorder may also be other portable devices, which need to include functions such as a camera and a recording device.

[0046] A method for intelligently supervising power supply service personnel in on-site communication with users on power demand, based on a work recorder, constructs a system for intelligently supervising the content and method of the response of power supply service personnel to the power demand raised by the user at the power supply service site, automatically identifies the power supply service personnel and the user through the on-site personnel identity recognition module, automatically converts the speech of the on-site personnel dialogue content into text through the speech-to-text module, finds the feature vector through the text-to-feature vector module, performs similarity matching with the feature vector in the power demand question-and-answer script library through the language dialogue matching module, finds the power demand and the corresponding demand response script that best match the on-site dialogue content and the on-site dialogue content in the power demand question-and-answer script library, and automatically identifies whether there is a demand related to electricity in the content of the communication between the user and the power supply service personnel during the power supply service process through comparison; once it is detected that the user in the dialogue has raised an electricity demand, the corresponding response content of the power supply service personnel will be supervised and judged according to the service specifications of the State Grid Corporation; if it is found that the response content of the power supply service personnel is wrong or the response content is incomplete, the early warning mechanism will be immediately triggered through the early warning module, and the power supply service personnel will be immediately reminded to standardize their service scripts to ensure service quality.

[0047] In the embodiment, the present invention can accurately capture the user demands during the entire service process, and monitor the content and method of the power supply service personnel's response to the user demands in real time, effectively improving the service quality and user satisfaction, and also helping the State Grid Corporation to establish a good service image. The specific steps are as follows:

[0048] The method specifically comprises the following steps:

[0049] Step 1: Establish a Q&A database for electricity demand;

[0050] According to the power supply service specifications of the State Grid and the historical user demands received by the State Grid 95598 hotline, a Q&A script library for power supply service questions and answers is established; the content of the Q&A script library for power demand is in the form of one question and one answer, and the relevant content is the power demand and the response to the demand; a text library of user power demand and a text library of corresponding demand responses of power supply service personnel are established respectively;

[0051] Step 2: Establish an on-site personnel identification module;

[0052] First, the on-site voice is collected through the work recorder; then, the voice of the on-site power supply service personnel and other voices are identified using voiceprint biometrics technology; after that, the identity of the person in the conversation is determined in real time; that is, if the voice of the power supply service personnel is identified, the speaker of the voice is determined to be the power supply service personnel, and if the voice of the power supply service personnel is not identified, the speaker of the voice is determined to be the user;

[0053] Step 3: Create a speech-to-text module;

[0054] Establish a voice-to-text module for power supply service personnel and users at the service site; use the power supply service personnel's voice and user's voice obtained in step 2 to carry out voice-to-text conversion for the on-site power supply service personnel and users based on the Whisper model. By establishing a voice-to-text module, the voices of users and on-site service personnel are converted into user text A and power supply service personnel text B at the service site for subsequent text library matching;

[0055] Step 4: Establish a text-to-feature vector module;

[0056] Through the user text A and the power supply service personnel text B output in step 3, the important feature vectors in the two texts are found respectively; by establishing a text-to-feature vector module, the user text A and the power supply service personnel text B are converted into feature vectors at the service site for subsequent dialogue question-answer matching;

[0057] Step 5: Establish a language dialogue matching module;

[0058] The feature vector of the power supply service personnel text B and the feature vector of the user text A extracted in step 4 are matched with the feature vectors of the user power demand text library and the demand answer text library in the power demand question and answer script library respectively; after feature vector matching similarity matching, the power demand and the corresponding demand answer script that best match the on-site conversation content are found in the power demand question and answer script library;

[0059] Feature vector matching similarity refers to measuring the similarity between two or more feature vectors by calculating the distance or similarity between them in fields such as computer vision or natural language processing;

[0060] Step 6: Establish an early warning module;

[0061] When it is determined through step 5 that the power supply service personnel have not properly responded to the user's service request, a warning signal is quickly issued to the power supply service personnel, requiring them to carefully and comprehensively answer the user's related demand questions to avoid inappropriate handling of user requests.

[0062] The power supply service site refers to the scene where staff provide electricity services to users, and the scope is not limited to power supply business halls, power consumption sites, etc.

[0063] The 95598 hotline in step 1 refers to the customer service hotline of State Grid Corporation of China.

[0064] The voiceprint biometrics in step 2 is a technology that verifies identity by analyzing personal voice characteristics. The process of establishing voiceprint recognition for power supply service personnel is divided into two stages: voiceprint modeling for power supply service personnel and voiceprint recognition for power supply service personnel:

[0065] 1) Voiceprint modeling of power supply service personnel

[0066] The power supply service personnel read a set of specific sentences or digital strings according to the instructions; then, the collected audio data is pre-processed using an audio processing device to ensure the audio quality; secondly, the acoustic features of the power supply service personnel are extracted, including pitch, formant, motion characteristics of the vocal organs, etc.; feature extraction is usually completed through acoustic models such as linear predictive coding, Mel frequency cepstral coefficients, etc.; finally, a voiceprint template of the power supply service personnel is established;

[0067] In voiceprint recognition, the statistical model commonly used for modeling is usually the Gaussian Mixture Model (GMM).

[0068] GMM is a mixture of multiple Gaussian distributions; first, for multidimensional Gaussian distribution, its probability density function is determined by the mean vector μ and the covariance matrix ∑;

[0069]

[0070] |∑| is the determinant of the covariance matrix;

[0071] GMM can be obtained by mixing multiple Gaussian distributions:

[0072]

[0073] The above formula represents a Gaussian mixture model with k Gaussian components, where π k is the mixing weight of the kth component, satisfying And π k >0; Ν(Xμ k ,Σ k ) is the probability density function of the kth Gaussian component;

[0074] Given a set of data X = {X1, X2, ..., X N}, we need to find the parameter set with the highest probability of data occurrence θ={π1...π k ,μ1...μ k ,∑1,...,∑ k},Right now:

[0075]

[0076] where L(θ;X) is the log-likelihood function:

[0077]

[0078] In GMM, each data point x iis considered to be drawn from one of k possible Gaussian distributions, but which distribution is not directly observed. Therefore, a latent variable z is needed i To represent, where z i is a k-dimensional binary vector with only one element being 1 and the rest being 0, indicating that x i From which component?

[0079] Due to the existence of hidden variables, it is difficult to directly maximize the log-likelihood function; therefore, the expectation-maximization (EM) algorithm is usually used to iteratively solve the problem;

[0080] E step (expectation step): Calculate the posterior probability γ(Z ik ), i.e. responsibility;

[0081]

[0082] M step (maximization step): Update the model parameters based on the responsibility obtained in the E step to maximize the log-likelihood function;

[0083] Update the mixing weight π k :

[0084]

[0085] Update the mean vector μ k :

[0086]

[0087] Update the covariance matrix ∑k:

[0088]

[0089] Repeat the E and M steps until the change in the log-likelihood function is less than a certain threshold, or the predetermined maximum number of iterations is reached;

[0090] 2) Voiceprint recognition for power supply service personnel

[0091] By verifying the voiceprint of the power supply service personnel and preprocessing the on-site audio data; then, using the on-site audio data to extract acoustic features, such as the logarithmic Mel spectrum; by preprocessing the audio signal, performing a fast Fourier transform on the preprocessed audio signal, passing the Fourier transformed spectrum through a Mel filter bank to obtain the energy of the Mel frequency band, and taking the logarithm of the energy of the Mel frequency band, the logarithmic Mel spectrum can be obtained;

[0092] After that, the newly extracted features are compared with the established voiceprint template; finally, if the matching degree is higher than the predetermined threshold, the person is verified as a power supply service personnel; otherwise, the person is verified as a customer;

[0093] By identifying the voiceprints of power supply service personnel in on-site speech, the identity of the personnel during the conversation can be determined to prepare for subsequent work.

[0094] The flow chart of the on-site personnel identification module is as follows: Figure 3 shown.

[0095] The Whisper model of step 3, its architecture (see Figure 4 ) is based on Transformer, with an 80-channel log-mel spectrogram as input; the encoder of the Whisper model consists of two convolutional layers with a kernel size of 3, sinusoidal positional encoding, and a series of stacked transformer blocks; the decoder also uses the learned positional embedding and uses the same number of transformer blocks as the encoder; the model adopts a multi-task training format, with a single decoder training model to perform various speech processing tasks; multi-task training is achieved by conditioning the decoder on an input token sequence that specifies a task and the expected output format;

[0096] The speech of on-site power supply service personnel and users is processed into logarithmic Mel-spectrograms respectively, and the whisper model is used as input to output the speech text of power supply service personnel and user speech text.

[0097] The process of the speech-to-text module is as follows Figure 5 shown.

[0098] The text-to-feature-vector-related model algorithm of the text-to-feature-vector module in step 4 adopts the BERT model. The BERT model is a pre-trained language model based on the Transformer architecture. The performance of the BERT model in multiple natural language processing tasks has been significantly improved, especially in question answering, text classification, named entity recognition, etc. The BERT model uses the encoder-decoder part of a multi-layer Transformer as its main architecture, see Figure 6 .

[0099] By establishing this module, at the service site, the texts of on-site power supply service personnel and users are converted into feature vectors for subsequent conversation question and answer matching.

[0100] The flowchart of the text-to-feature vector module is as follows: Figure 7 shown.

[0101] The step 5 establishes a language dialogue matching module. Feature vector matching similarity refers to measuring the similarity between two or more feature vectors by calculating the distance or similarity between them in the fields of computer vision or natural language processing.

[0102] Cosine Similarity is a commonly used similarity calculation that measures the similarity between features by measuring the cosine value of the angle between two vectors.

[0103]

[0104] Among them: A and B are two vectors; A·B represents the dot product (inner product) of vectors A and B, that is where a i and b i are the values ​​of vector A and B in the i-th dimension respectively; / / A / / ​​and / / B / / represent the modulus (length) of vector A and B respectively.

[0105] The present invention realizes real-time supervision of the communication content between power supply service personnel and users at the power supply service site through a work recorder and an intelligent algorithm model, and can automatically identify whether there are demands related to electricity use in the content of communication between users and power supply service personnel during the power supply service process. Once it is detected that the user in the conversation has raised a demand for electricity use, the corresponding reply content of the power supply service personnel will be supervised and judged in strict accordance with the service specifications of the State Grid Corporation. If it is found that the reply content of the power supply service personnel is wrong or the reply content is incomplete, the early warning mechanism will be triggered immediately, and the power supply service personnel will be reminded immediately, requiring them to standardize their service words to ensure service quality.

Claims

1. A device for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that: It is used in the scenario where the on-site staff of the State Grid power supply service respond to the power consumption demands of on-site users. Based on the work recorder, a system is built to intelligently supervise the content and method of the response of the power supply service personnel to the power consumption demands raised by the users at the power supply service site. The system includes a power consumption demand question and answer script library, a field personnel identity recognition module, a speech-to-text module, a text-to-feature vector module, a language dialogue matching module and an early warning module. The real-time supervision of the communication content between the service personnel and the users at the power supply service site is realized through the work recorder and the intelligent algorithm model; the power supply service personnel and users are automatically identified through the on-site personnel identity recognition module, the speech of the on-site personnel conversation content is automatically converted into text through the speech-to-text module, the feature vector is found through the text-to-feature vector module, and the similarity is matched with the feature vector in the power consumption demand question and answer script library through the language dialogue matching module. The power consumption demand and the corresponding demand response script that best match the on-site conversation content are found in the power consumption demand question and answer script library. By comparison, it is automatically identified whether there are demands related to electricity consumption in the content of the communication between the user and the power supply service personnel during the power supply service process; Once it is detected that the user in the conversation has made a request for electricity use, the corresponding response of the power supply service personnel will be monitored and judged according to the service specifications of the State Grid Corporation; If it is found that the reply of the power supply service personnel contains errors or is incomplete, the early warning mechanism will be immediately triggered through the early warning module to give instant reminders to the power supply service personnel, requiring them to standardize their service words and ensure service quality.

2. A method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that: Based on the work recorder, a system is built to intelligently supervise the content and method of the responses of power supply service personnel to users' power demand at the power supply service site. The power supply service personnel and users are automatically identified through the on-site personnel identity recognition module, and the speech of the on-site personnel's conversation content is automatically converted into text through the speech-to-text module. The feature vector is found through the text-to-feature vector module, and the similarity is matched with the feature vector in the power demand question and answer script library through the language dialogue matching module. The power demand and the corresponding demand response script that best match the on-site conversation content are found in the power demand question and answer script library. By comparison, it is automatically identified whether there are demands related to electricity in the content of the communication between the user and the power supply service personnel during the power supply service process; Once it is detected that the user in the conversation has made a request for electricity use, the corresponding response of the power supply service personnel will be monitored and judged according to the service specifications of the State Grid Corporation; If it is found that the reply of the power supply service personnel contains errors or is incomplete, the early warning mechanism will be immediately triggered through the early warning module to give instant reminders to the power supply service personnel, requiring them to standardize their service words and ensure service quality.

3. According to claim 2, a method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that The method specifically comprises the following steps: Step 1: Establish a Q&A database for electricity demand; According to the power supply service specifications of the State Grid and the historical user demands received by the State Grid 95598 hotline, a Q&A script library for power supply service questions and answers is established; the content of the Q&A script library for power demand is in the form of one question and one answer, and the relevant content is the power demand and the response to the demand; a text library of user power demand and a text library of corresponding demand responses of power supply service personnel are established respectively; Step 2: Establish an on-site personnel identification module; First, the on-site voice is collected through the work recorder; then, the voice of the on-site power supply service personnel and other voices are identified using voiceprint biometrics technology; after that, the identity of the person in the conversation is determined in real time; that is, if the voice of the power supply service personnel is identified, the speaker of the voice is determined to be the power supply service personnel, and if the voice of the power supply service personnel is not identified, the speaker of the voice is determined to be the user; Step 3: Create a speech-to-text module; Establish a voice-to-text module for power supply service personnel and users at the service site; use the power supply service personnel's voice and user's voice obtained in step 2 to carry out voice-to-text conversion for the on-site power supply service personnel and users based on the Whisper model. By establishing a voice-to-text module, the voices of users and on-site service personnel are converted into user text A and power supply service personnel text B at the service site for subsequent text library matching; Step 4: Establish a text-to-feature vector module; Through the user text A and the power supply service personnel text B output in step 3, the important feature vectors in the two texts are found respectively; by establishing a text-to-feature vector module, the user text A and the power supply service personnel text B are converted into feature vectors at the service site for subsequent dialogue question-answer matching; Step 5: Establish a language dialogue matching module; The feature vector of the power supply service personnel text B and the feature vector of the user text A extracted in step 4 are used to match the feature vectors of the user power demand text library and the demand answer text library in the power demand question and answer script library respectively; After feature vector matching and similarity matching, the electricity demand and corresponding answering script that best match the on-site conversation content are found in the electricity demand question and answer script library; Step 6: Establish an early warning module; When it is determined through step 5 that the power supply service personnel have not properly responded to the user's service request, a warning signal is quickly issued to the power supply service personnel, requiring them to carefully and comprehensively answer the user's related demand questions to avoid inappropriate handling of user requests.

4. According to claim 3, a method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that: The voiceprint biometrics in step 2 is a technology that verifies identity by analyzing personal voice characteristics. The process of establishing voiceprint recognition for power supply service personnel is divided into two stages: voiceprint modeling for power supply service personnel and voiceprint recognition for power supply service personnel: 1) Voiceprint modeling of power supply service personnel The power supply service personnel read a set of specific sentences or digital strings according to the instructions; then, the collected audio data is pre-processed using an audio processing device to ensure the audio quality; secondly, the acoustic characteristics of the power supply service personnel are extracted; Finally, establish a voiceprint template for power supply service personnel; 2) Voiceprint recognition for power supply service personnel By verifying the voiceprint of the power supply service personnel and preprocessing the on-site audio data; then, using the on-site audio data to extract acoustic features; after that, comparing the newly extracted features with the established voiceprint template; finally, if the degree of match is higher than the predetermined threshold, the person is verified as a power supply service personnel; otherwise, the person is verified as a customer; by identifying the voiceprint of the power supply service personnel in the on-site voice, the identity of the person during the conversation is determined to prepare for subsequent work.

5. According to claim 3, a method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that: The Whisper model of step 3 is based on the Transformer architecture, and the input is an 80-channel logarithmic Mel-spectrogram; the encoder of the Whisper model consists of two convolutional layers with a convolution kernel size of 3, sinusoidal position encoding, and a series of stacked transformer blocks; the decoder also uses the learned position embedding and uses the same number of transformer blocks as the encoder; the model adopts a multi-task training format, and a single decoder is used to train the model to perform various speech processing tasks; Multi-task training is achieved by conditioning the decoder on a sequence of input tokens that specifies the task and the desired output format; The speech of on-site power supply service personnel and users is processed into logarithmic Mel-spectrograms respectively, and the whisper model is used as input to output the speech text of power supply service personnel and user speech text.

6. According to claim 6, a method for intelligently supervising power supply service personnel to communicate with users on-site about power demand, characterized in that: The text-to-feature-vector related model algorithm of the text-to-feature-vector module in step 4 adopts the BERT model.

Images