An audio analysis-based accompanying diagnosis service quality evaluation method and device

Abstract

The application discloses a kind of based on audio analysis's accompanying diagnosis service quality evaluation method and device, belong to artificial intelligence and speech signal processing technical field.The method includes: obtaining audio data in the process of accompanying diagnosis service, which represents the interaction between accompanying doctor and user;Audio data is processed to extract acoustic features, emotional features and semantic features;Based on the multi-dimensional features extracted, score according to the preset evaluation dimensions such as service attitude, communication clarity, etc.;And according to the context information such as service type, user label, dynamically adjust the weight of each evaluation dimension;Finally, based on the adjusted weight and the dimension score, calculate the comprehensive score, generate structured service quality evaluation result.The application realizes the automation, objectivity and refinement of the evaluation of accompanying diagnosis service quality through multi-dimensional feature fusion and dynamic weight adjustment, can adapt to different service scenarios and user needs, and provide targeted improvement suggestions, effectively improve the accuracy and practicality of evaluation.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and speech signal processing technology, and in particular to a method and apparatus for evaluating the quality of escort services based on audio analysis. Background Technology

[0002] As an emerging health service model, the quality of escort services directly impacts user experience, platform reputation, and even user health and safety. Currently, the evaluation of escort service quality primarily relies on subjective user ratings after the service, random checks by platform operators, or follow-up phone calls from customer service personnel.

[0003] However, these traditional evaluation methods have limitations. First, user ratings are highly subjective and easily influenced by non-service factors such as user emotions and rating habits. Furthermore, user participation is generally low, leading to significant data sample bias and making it difficult to objectively and comprehensively reflect the true service quality. Second, platform spot checks and manual follow-ups not only have extremely limited coverage, easily overlooking service orders with potential risks, but also suffer from inconsistent evaluation standards, making it difficult to guarantee the objectivity and consistency of quality inspection results. In addition, manual quality inspection methods are costly and inefficient, unable to achieve full coverage of the massive number of service orders.

[0004] During the service process, the voice interaction between the escort and the user is the most direct reflection of service quality, containing a wealth of analyzable information, such as the fluency of communication, the patience of the tone, and the confirmation of key information. Although voice-based quality inspection solutions already exist in areas such as customer service centers, these solutions typically focus on compliance checks of standard scripts or the statistical analysis of specific keyword hits. Their evaluation rules are relatively fixed and cannot adapt to the complex and ever-changing service content and personalized user needs in the escort service scenario.

[0005] In summary, existing technologies for assessing the quality of escort services generally suffer from drawbacks such as reliance on subjective judgment, high labor costs, lack of objective quantitative indicators for the service process, rigid assessment rules, and inability to adapt to diverse service scenarios. Summary of the Invention

[0006] Given that the quality assessment of escort services in existing technologies mainly relies on subjective scoring or manual sampling, which has problems such as poor objectivity, high cost, single assessment dimensions and rigid rules, this invention aims to provide a method and device for assessing the quality of escort services based on audio analysis, so as to achieve automated, objective and refined assessment of the quality of escort services.

[0007] This invention provides a method for evaluating the quality of escort services based on audio analysis, comprising: acquiring audio data representing the interaction between the escort and the user during the escort service; processing the audio data to extract acoustic features reflecting the physical properties of speech; processing the audio data to identify the emotional state information of the participants in the dialogue as emotional features; extracting semantic features representing the dialogue content based on the audio data; scoring the quality of the escort service according to multiple preset evaluation dimensions based on the extracted multi-dimensional audio features, obtaining scores for each evaluation dimension; dynamically adjusting the weights of each evaluation dimension according to contextual information associated with the escort service; calculating a comprehensive service quality score based on the adjusted weights and the scores of each evaluation dimension, and generating a service quality evaluation result based on the comprehensive service quality score.

[0008] In one embodiment, before processing the audio data to extract acoustic features reflecting the physical properties of speech, the method further includes preprocessing the audio data, the preprocessing including at least one of noise reduction, speech activity detection, and speaker separation.

[0009] In one implementation, the preprocessing includes speaker separation; and the step of extracting the acoustic features and / or emotional features includes extracting or statistically analyzing the acoustic features and / or emotional features of the caregiver and the user, respectively, based on the results of speaker separation.

[0010] In one implementation, the context information includes at least one of service type, user tags, and user historical rating information.

[0011] In one implementation, dynamically adjusting the weights of each evaluation dimension based on contextual information associated with the accompanying medical service includes: increasing the weight of the communication clarity and service professionalism dimension when the contextual information includes the service type as accompanying medical treatment; and increasing the weight of the communication clarity and service attitude dimension when the contextual information includes the user tag as elderly.

[0012] In one implementation, the preset multiple evaluation dimensions include at least two of service attitude, communication clarity, service professionalism, and user emotional satisfaction.

[0013] In one implementation: when the preset multiple evaluation dimensions include communication clarity, the acoustic features include at least one of speech rate, volume, pitch, pause frequency, and speech duration percentage, and are used to calculate the score for the communication clarity dimension; when the preset multiple evaluation dimensions include service attitude and / or user emotional satisfaction, the emotional features include the duration percentage of the companion and / or user's pleasant, impatient, or dissatisfied emotions, and are used to calculate the score for the service attitude and / or user emotional satisfaction dimension; when the preset multiple evaluation dimensions include service attitude and / or service professionalism, the semantic features include the detection results of polite language, negative communication content, and key process confirmation, and are used to calculate the score for the service attitude and / or service professionalism dimension.

[0014] In one implementation, extracting semantic features representing the dialogue content based on the audio data includes: converting the audio data into text data using automatic speech recognition technology; and performing natural language processing on the text data to extract the semantic features.

[0015] In one implementation, the service quality assessment result is a structured assessment report, which includes the overall score, scores for each assessment dimension, and targeted improvement suggestions generated based on the problem items identified in the score results.

[0016] This invention also provides an audio analysis-based device for evaluating the quality of escort services, comprising: an acquisition module configured to acquire audio data representing the interaction between the escort and the user during the escort service; a feature extraction module configured to process the audio data to extract acoustic features reflecting the physical properties of speech, identify the emotional state information of the participants in the dialogue as emotional features, and extract semantic features representing the content of the dialogue; a scoring module configured to score the quality of the escort service according to multiple preset evaluation dimensions based on the extracted multi-dimensional audio features, and obtain scores for each evaluation dimension; a weight adjustment module configured to dynamically adjust the weights of each evaluation dimension according to contextual information associated with the escort service; and a result generation module configured to calculate a comprehensive service quality score based on the adjusted weights and the scores of each evaluation dimension, and generate a service quality evaluation result based on the comprehensive service quality score.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. Objectively Quantify Service Quality: By integrating acoustic, emotional, and semantic features for comprehensive analysis, traditional soft indicators such as service attitude and communication quality are transformed into calculable and comparable objective data, effectively avoiding the subjectivity and bias of manual evaluation and making the evaluation results more fair and reliable.

[0018] 2. Achieve full-scale automated evaluation: The method and apparatus provided by this invention can automate the entire process from audio data acquisition to evaluation report generation, enabling undifferentiated quality evaluation of all escort service orders, achieving 100% full coverage, while reducing the cost and management burden of manual quality inspection.

[0019] 3. Comprehensive and In-Depth Evaluation Dimensions: This invention not only focuses on results but also on the service process. By constructing an evaluation system that includes multiple dimensions such as service attitude, communication clarity, and service professionalism, it provides refined measurement of key behaviors and performance during the service process, resulting in a more comprehensive and in-depth evaluation.

[0020] 4. Dynamic Adaptive Evaluation Rules: This invention introduces a dynamic weight adjustment mechanism based on contextual information, which can automatically adjust the evaluation focus according to factors such as service type and user characteristics. This allows the same evaluation system to adapt to the differentiated needs of different service scenarios, improving the relevance, accuracy, and practicality of the evaluation results.

[0021] 5. Driving continuous improvement in service quality: By generating structured evaluation reports that include problem identification and targeted improvement suggestions, it provides companions with clear and feasible directions for improvement, which helps to form a virtuous cycle of evaluation-feedback-improvement, thereby systematically promoting the continuous improvement of the overall service level. Attached Figure Description

[0022] Figure 1 A flowchart of an audio analysis-based method for evaluating the quality of escort services provided in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the structure of the audio analysis-based escort service quality assessment device provided in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of an audio acquisition and processing flow for a medical escort service provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0026] Figure 1 The flowchart below shows the audio analysis-based method for evaluating the quality of escort services provided by this invention. Figure 1 Each step in the process will be explained in detail.

[0027] Step S110: Obtain audio data representing the interaction between the escort and the user during the escort service.

[0028] This step first involves acquiring audio data representing the interaction between the companion and the user during the companionship service.

[0029] Step S120: Process the audio data to extract acoustic features that reflect the physical properties of speech, identify the emotional state information of the dialogue participants as emotional features, and extract semantic features that represent the content of the dialogue.

[0030] After acquiring the audio data, the system processes it, extracting multi-dimensional features in parallel or serial manner. Specifically, it extracts acoustic features reflecting the physical properties of speech, identifies the emotional states of the participants in the dialogue as affective features, and extracts semantic features representing the content of the dialogue based on the audio data. These multi-dimensional features provide an objective data foundation for subsequent quantitative evaluation.

[0031] Step S130: Based on the extracted multi-dimensional audio features, score the quality of the escort service according to multiple preset evaluation dimensions to obtain the score of each evaluation dimension.

[0032] Based on these extracted multi-dimensional audio features, the system scores the quality of the escort service according to multiple preset evaluation dimensions, thereby obtaining specific scores for each evaluation dimension.

[0033] Step S140: Dynamically adjust the weights of each assessment dimension based on the context information associated with the escort service.

[0034] To make the assessment more targeted, the system will dynamically adjust the weight of each assessment dimension based on the context information associated with the accompanying medical service.

[0035] Step S150: Calculate the comprehensive service quality score based on the adjusted weights and scores of each evaluation dimension, and generate the service quality evaluation result based on the comprehensive service quality score.

[0036] A comprehensive service quality score is calculated based on the adjusted weights and scores for each evaluation dimension, and the final service quality assessment result is generated based on this comprehensive score. This complete technological loop, through automated processing and dynamic adjustment, enables an objective, comprehensive, and flexible assessment of service quality.

[0037] In one implementation, the method further includes preprocessing the audio data before extracting acoustic, emotional, and semantic features. The preprocessing steps may include at least one of noise reduction, speech activity detection, and speaker separation. Noise reduction effectively removes interference from environmental noise for subsequent analysis; speech activity detection accurately identifies valid speech segments in the audio and eliminates silent segments; and speaker separation clearly distinguishes the voices of the caregiver and the user. These preprocessing steps improve the accuracy and reliability of subsequent feature extraction, thereby ensuring the accuracy of the final evaluation results.

[0038] In one implementation, when preprocessing includes speaker separation, the step of extracting acoustic and / or emotional features can be based on the results of speaker separation, extracting or statistically analyzing the acoustic and / or emotional features of the caregiver and the user separately. For example, the caregiver's speech rate and volume, and the user's emotional state distribution can be calculated separately. This distinction allows the evaluation to focus on the caregiver's service behavior while using user feedback as an independent evaluation dimension, avoiding confusion between the two, thereby enabling a more refined and targeted analysis and evaluation of both parties in the service interaction.

[0039] In one implementation, the contextual information used to dynamically adjust weights may include at least one of service type, user tags, and user historical evaluation information. Service types include things like accompanying someone to a doctor's appointment or examination; user tags include things like "elderly" or "child"; and user historical evaluation information reflects the user's past service experience preferences. Introducing this contextual information allows the evaluation model to break free from the constraints of fixed rules and adaptively adjust according to specific scenarios, improving the practicality and accuracy of the evaluation.

[0040] In one implementation, the rules for dynamically adjusting weights can be preset. For example, when the context information includes the service type as "medical escort," the system can automatically increase the weights of the two evaluation dimensions: communication clarity and service professionalism, because accurate information delivery is crucial in such services. When the context information includes the user tag as elderly, the system can increase the weights of the dimensions of communication clarity (e.g., focusing on whether the speech rate is slowed down) and service attitude (e.g., focusing on the level of patience) to better meet the needs of the specific user group. This dynamic weight adjustment based on specific scenarios allows the evaluation focus to automatically align with the core service needs, thereby yielding more valuable evaluation results.

[0041] In one implementation, to achieve a comprehensive evaluation, the pre-defined evaluation dimensions may include at least two of the following: service attitude, communication clarity, service professionalism, and user emotional satisfaction. These dimensions characterize the essence of service quality from different perspectives: service attitude focuses on the emotional experience during the interaction process; communication clarity focuses on the effectiveness of information delivery; service professionalism focuses on the standardization of the service process; and user emotional satisfaction directly reflects the user's level of acceptance. By comprehensively considering these dimensions, a three-dimensional and comprehensive service quality profile can be constructed.

[0042] In one implementation, to associate abstract evaluation dimensions with specific audio features, this invention defines a feature-to-score mapping relationship. When the preset evaluation dimension includes communication clarity, the extracted acoustic features may include at least one of speech rate, volume, pitch, pause frequency, and speech duration percentage, and are used to calculate the score for the communication clarity dimension. When the preset evaluation dimension includes service attitude and / or user emotional satisfaction, the extracted emotional features may include the duration percentage of the companion and / or user's pleasant, impatient, or dissatisfied emotions, and are used to calculate the score for the corresponding dimension. Similarly, when the preset evaluation dimension includes service attitude and / or service professionalism, the extracted semantic features may include the detection results of polite language, negative communication content, and key process confirmations, and are used to calculate the score for the corresponding dimension. In this way, the underlying features are effectively connected with the high-level dimensions, realizing the transformation from raw audio to quantitative scoring.

[0043] In one implementation, the specific process of extracting semantic features may include: first, converting audio data into text data using automatic speech recognition technology; and then, performing natural language processing on the obtained text data to extract semantic features. For example, polite expressions can be detected using keyword matching or text classification models, negative communication can be identified using sentiment analysis models, and key process confirmation can be determined using intent recognition models. This two-stage processing approach fully utilizes mature speech recognition and natural language processing technologies, ensuring the accuracy and richness of semantic feature extraction.

[0044] In one implementation, the final service quality assessment result can be a structured assessment report. To provide guidance, this report may include a comprehensive score, scores for each assessment dimension, and targeted improvement suggestions generated based on the identified problem items. For example, when the score for a certain dimension falls below a preset threshold, the system can automatically mark it as a problem item and match corresponding improvement suggestions from a pre-built knowledge base. This design not only provides assessment conclusions but also indicates directions for improvement, thus constructing a closed loop for service quality enhancement: assessment-feedback-improvement.

[0045] After introducing the audio analysis-based method for evaluating the quality of escort services provided by this invention, the following section refers to... Figure 2 The accompanying medical service quality assessment device based on audio analysis provided by the present invention will be described.

[0046] refer to Figure 2 The quality assessment of escort services based on audio analysis includes: The acquisition module 210 is configured to acquire audio data representing the interaction between the escort and the user during the escort service. The feature extraction module 220 is configured to process audio data to extract acoustic features that reflect the physical properties of speech, identify the emotional state information of dialogue participants as emotional features, and extract semantic features that characterize the content of the dialogue. The scoring module 230 is configured to score the quality of the escort service based on the extracted multi-dimensional audio features and according to multiple preset evaluation dimensions, and obtain the score of each evaluation dimension. The weight adjustment module 240 is configured to dynamically adjust the weights of each evaluation dimension based on contextual information associated with the escort service. The results generation module 250 is configured to calculate a comprehensive service quality score based on the adjusted weights and scores of each evaluation dimension, and generate a service quality assessment result based on the comprehensive service quality score.

[0047] In one embodiment, the device may further include a preprocessing module. This preprocessing module preprocesses the acquired audio data before it is processed by the feature extraction module. The preprocessing may include at least one of noise reduction, speech activity detection, and speaker separation. This preprocessing module provides higher-quality input data for the subsequent feature extraction module, thereby improving the overall evaluation accuracy of the device.

[0048] In one implementation, when the preprocessing module performs speaker separation, the feature extraction module can be configured to extract or statistically analyze the acoustic and / or emotional features of the caregiver and user, respectively, based on the speaker separation results. This design enables the device to perform more granular analysis; for example, the feature extraction module can output the emotional distribution of the caregiver and the user separately, and the scoring module can then calculate service attitude scores and user emotional satisfaction scores accordingly, achieving independent evaluation of the status of both parties.

[0049] In one implementation, the contextual information used by the weight adjustment module may include at least one of service type, user tags, and user historical evaluation information. This enables the device to possess scene awareness and adaptive capabilities, allowing it to flexibly adjust the evaluation strategy according to different service scenarios and user characteristics, and output more realistic evaluation results.

[0050] In one implementation, the weight adjustment module can have a built-in set of adjustment rules. For example, the module is configured to automatically increase the weight of the communication clarity and service professionalism dimensions when the received context information includes the service type of "accompanying a patient to a doctor"; and automatically increase the weight of the communication clarity and service attitude dimensions when the context information includes the user tag of "elderly". In this way, the device's evaluation logic can dynamically align with service needs, achieving intelligent and personalized quality assessment.

[0051] In one implementation, to achieve a comprehensive evaluation, the pre-defined evaluation dimensions used by the scoring module may include at least two of the following: service attitude, communication clarity, service professionalism, and user emotional satisfaction. These dimensions ensure that the device can comprehensively assess service quality from multiple perspectives, resulting in a more holistic and complete set of conclusions.

[0052] In one implementation, regarding the association between specific features and dimensions, when the preset evaluation dimension includes communication clarity, the acoustic features extracted by the feature extraction module may include at least one of speech rate, volume, pitch, pause frequency, and speech duration percentage, and are used by the scoring module to calculate the score for the communication clarity dimension. When the preset evaluation dimension includes service attitude and / or user emotional satisfaction, the emotional features extracted by the feature extraction module may include the duration percentage of the companion and / or user's feelings of pleasure, impatience, or dissatisfaction, and are used by the scoring module to calculate the score for the corresponding dimension. When the preset evaluation dimension includes service attitude and / or service professionalism, the semantic features extracted by the feature extraction module may include the detection results of polite language, negative communication content, and key process confirmations, and are used by the scoring module to calculate the score for the corresponding dimension. This explicit correspondence ensures the clarity and interpretability of the device's evaluation logic.

[0053] In one implementation, the feature extraction module can be configured to: first convert the acquired audio data into text data using automatic speech recognition technology; and then perform natural language processing on the text data to extract semantic features. This modular design allows the device to integrate industry-leading speech and natural language processing engines to ensure effective semantic feature extraction.

[0054] In one implementation, to make the evaluation results more practical, the service quality evaluation results generated by the results generation module can be a structured evaluation report. This report includes not only a comprehensive score and scores for each evaluation dimension, but also problem items identified based on the scoring results and improvement suggestions generated for these problem items. This output format makes the evaluation results not just a score, but also an actionable improvement guide, providing strong support for the continuous improvement of service quality.

[0055] The following will be combined with the appendix Figure 3 This invention describes the architecture and data flow process of an audio analysis-based system for evaluating the quality of escort services provided by this invention. The system can be implemented using a typical client / server (C / S) architecture. Figure 3 As shown, the client can be the companion's terminal device, such as a smartphone with a specific application installed. This application is responsible for collecting audio data during the service process, segmenting the audio, and uploading it to a cloud server. The core functional modules of the assessment system are deployed within the cloud server. After the audio data is uploaded, it first enters the audio preprocessing module, which performs noise reduction, speech activity detection, and speaker separation, outputting clean, labeled speech segments. Subsequently, the processed data is sent to the multi-dimensional feature extraction module, which can be further subdivided into three parallel sub-modules: acoustic, emotional, and semantic, extracting corresponding features for each. The extracted multi-dimensional features are structured and stored in a feature database. The scoring module, weight adjustment module, and result generation module read data from the feature database, complete a series of processes such as scoring, weight adjustment, comprehensive evaluation, and report generation, and finally store the service quality assessment results in the database or push them to the front end for display. The entire data flow starts with audio collection from the companion's terminal device, goes through a series of processes on the cloud server, and finally generates service quality assessment results, forming a complete closed loop.

[0056] To more clearly illustrate the core steps of the method of this invention, a basic method embodiment is provided below. First, the system acquires audio data representing the interaction between the companion and the user, collected by the companion's terminal device during a complete medical accompaniment service. Next, the system processes this audio data, extracting features in three dimensions: acoustic features reflecting the physical properties of speech, emotional state information identified by a model as emotional features, and semantic features representing the core content of the dialogue. After acquiring these multi-dimensional features, the system scores each feature based on several preset evaluation dimensions (e.g., service attitude and communication quality), obtaining a preliminary score for each dimension. Subsequently, the system acquires contextual information related to the service, such as the service type being medical accompaniment, and dynamically adjusts the weights of each evaluation dimension based on this information. Finally, the system uses the adjusted weights to perform a weighted summation of the scores for each dimension, calculating the final comprehensive service quality score and generating a service quality evaluation result containing this score. This embodiment describes the core technical logic of this invention, namely, achieving objective evaluation through multi-dimensional feature analysis and dynamic weight adjustment, corresponding to the maximum protection scope of this invention.

[0057] To more comprehensively demonstrate the preferred embodiments of the present invention, the following will be described in conjunction with the accompanying drawings. Figure 1 and attached Figure 3 This document details a complete evaluation process that integrates multiple technical features. The process begins with the acquisition of audio data. After the companionship service commences, the application on the companion's terminal device acquires audio data at a 16kHz sampling rate, 16-bit quantization precision, and in mono format, buffering the audio data in segments every 30 seconds. This parameter setting ensures the integrity of the voice information, while the segmentation mechanism facilitates real-time uploading and processing. After the service concludes, all segments are merged and uploaded to the cloud server.

[0058] like Figure 3 As shown, after receiving audio data, the cloud server first processes it using an audio preprocessing module. This includes noise reduction using a deep learning model, Voice Activity Detection (VAD) to identify valid speech segments, and speaker separation to distinguish between the voices of the caregiver and the user. The processed data is then sent to a multi-dimensional feature extraction module. In this module, the system extracts acoustic features, such as calculating the Mel-Frequency Cepstral Coefficients (MFCC) and fundamental frequency F0 for each speech segment; extracts emotional features, for example, by inputting the speech segment into an emotion state classification model fine-tuned based on a pre-trained speech model (wav2vec 2.0) to identify its emotional state; and extracts semantic features, such as using the Whisper model for automatic speech recognition, converting the audio into timestamped text for further analysis. All extracted features are stored in a feature database.

[0059] Next, as Figure 1 As shown, the evaluation process enters the comprehensive evaluation stage. First, the system inputs multi-dimensional features from the feature database. Then, it proceeds to the calculation step for the scores of each dimension. Taking the service attitude dimension as an example, its score can be calculated according to the following formula: Service attitude score = 100 × [1 - (Percentage of time spent in a patient's negative emotional state × ... - (Number of times volume exceeded threshold / service duration ×) )]+Frequency of polite expressions × .in, , , For example, the preset weighting coefficients. It can be set to 0.6. It can be set to 0.1. It can be set to 2. This formula quantifies the impact of negative emotions, excessive volume, and polite language on service attitude.

[0060] For the communication clarity dimension, the score can be calculated using the formula: Communication Clarity Score = 100 × [1 - (Speech Rate Deviation Rate × ... )]×[1-(volume deviation rate× )]×[1-(pause anomaly rate× The calculation involves the speech rate deviation rate, which is the degree of deviation between the actual speech rate and the target speech rate (e.g., 180 words / minute). , , These are weighting coefficients, which can be set to 0.5, 0.3, and 0.2 respectively. This formula aims to assess the appropriateness of speech rate, volume, and pauses. For the service professionalism dimension, the score can be calculated using the formula: Service Professionalism Score = 100 × (Key Process Confirmation Coverage × ... ) + (Accuracy rate of disease description × ) calculate, where and For the user emotional satisfaction dimension, the score can be calculated using the formula: User emotional satisfaction score = 100 × (Percentage of user's positive emotional state duration × ... + (Percentage of user's negative emotional state ×) ) calculate, where For positive weights (e.g., 1.0). It is a negative weight (e.g., -0.5).

[0061] After calculating the initial scores for each dimension, the system performs dynamic weight adjustments. The system obtains the context information of the current order, such as the service type being "medical accompaniment" and the user tag being "elderly." Assume the base weights are: service attitude 0.25, communication clarity 0.25, service professionalism 0.30, and user emotional satisfaction 0.20. According to the rules for medical accompaniment, the weights for communication clarity and service professionalism are each increased by 0.05; according to the rules for the elderly, the weights for communication clarity and service attitude are each increased by 0.05. After merging and adjustment, and normalization, the final weights may become: service attitude 0.30, communication clarity 0.30, service professionalism 0.30, and user emotional satisfaction 0.10.

[0062] Subsequently, the system calculates the comprehensive score based on the adjusted weights and scores of each dimension. The calculation formula is: Comprehensive score = Σ (score of each dimension × weight of that dimension) / Σ (weight of each dimension).

[0063] Finally, based on all the scoring and analysis results, the system generates a service quality assessment result in the form of a structured assessment report. This report clearly identifies problem items with scores below a specific threshold (e.g., 60 points) and matches them with a pre-defined improvement suggestion library to generate targeted improvement recommendations. For example, when the communication clarity score is low, it suggests appropriately slowing down the speaking speed and ensuring the volume is moderate. Through this complete process, this invention realizes an automated and intelligent assessment system from data to insights.

[0064] To further clarify the key technical modules of this invention, more in-depth implementation details are provided below. First, regarding the optimization of the emotion state recognition model in emotion feature extraction. In one embodiment of this invention, to improve the model's accuracy in the specific domain of escort services, the following optimization measures are taken: First, a powerful base model is selected, such as the wav2vec 2.0 model pre-trained on large-scale unlabeled speech data, because it has excellent speech representation capabilities. Next, real dialogue audio data in the escort service scenario is collected and labeled specifically. Professional annotators label the speech segments with emotions, including calm, joy, impatience, dissatisfaction, anger, anxiety, etc. During the annotation process, a comprehensive judgment is made in conjunction with the dialogue context, and a consistency check mechanism is established to ensure annotation quality. Then, a fully connected layer and an output layer for emotion classification are added to the wav2vec 2.0 model, and the entire model is fine-tuned using labeled domain data. During training, data augmentation strategies such as velocity perturbation, pitch perturbation, and noise injection are also used to improve the model's robustness and generalization ability. The model trained in this way can more accurately identify subtle emotional changes in conversations during medical visits compared to general emotion recognition models.

[0065] Secondly, the specific implementation of the dynamic weight adjustment mechanism is discussed. In one embodiment of this invention, the mechanism is implemented through a configurable weight adjustment rule base. This rule base is stored in key-value pairs, where the key is the trigger condition and the value is the corresponding weight adjustment amount. For example, a rule can be defined as: condition:service_type:accompaniment for medical treatment, adjustment:communication clarity:+0.05,service professionalism:+0.05. Another rule could be: condition:user_tag:elderly, adjustment:communication clarity:+0.05,service attitude:+0.05. When a service simultaneously meets multiple conditions, the system requires a fusion strategy. A weighted average strategy can be used, setting different priorities or fusion weights for different types of conditions (such as user tags, service types, and historical evaluations), and then summing the weighted adjustments for each condition to obtain the total adjustment amount. After adding the total adjustment amount to the base weights, normalization is performed again to ensure that the sum of all dimension weights is 1. Furthermore, to prevent imbalances in the evaluation results, the system sets upper and lower limits for the weights. For example, it stipulates that the weight of any dimension must not exceed 0.40 or be lower than 0.05. This refined implementation ensures the flexibility, robustness, and controllability of the dynamic weight adjustment mechanism.

[0066] Those skilled in the art will understand that the embodiments of the present invention described above can be implemented by hardware, software, or firmware. Regardless of the method used, the apparatus and methods described in the above embodiments can be implemented on a microprocessor or conventional computer by programming or other suitable means. Furthermore, the method steps of the present invention can be embodied as a set of instructions stored in a storage medium, which can be any type of computer-readable storage medium, such as read-only memory (ROM), random access memory (RAM), magnetic or optical recording media, etc.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the quality of escort services during medical visits based on audio analysis, characterized in that, include: Acquire audio data representing the interaction between the companion and the user during the companionship service; The audio data is processed to extract acoustic features that reflect the physical properties of speech, identify the emotional state information of the dialogue participants as emotional features, and extract semantic features that characterize the content of the dialogue. Based on the extracted multi-dimensional audio features, the quality of the escort service is scored according to multiple preset evaluation dimensions, and the scores of each evaluation dimension are obtained. The weights of each evaluation dimension are dynamically adjusted based on the context information associated with the accompanying medical service. A comprehensive service quality score is calculated based on the adjusted weights and scores of each evaluation dimension, and a service quality assessment result is generated based on the comprehensive service quality score.

2. The method according to claim 1, characterized in that, Before processing the audio data to extract acoustic features reflecting the physical properties of speech, the method further includes preprocessing the audio data, the preprocessing including at least one of noise reduction, speech activity detection, and speaker separation.

3. The method according to claim 2, characterized in that, The preprocessing includes speaker separation; and the step of extracting the acoustic features and / or emotional features includes extracting or statistically analyzing the acoustic features and / or emotional features of the caregiver and the user, respectively, based on the results of speaker separation.

4. The method according to claim 1, characterized in that, The context information includes at least one of service type, user tags, and user historical rating information.

5. The method according to claim 4, characterized in that, The step of dynamically adjusting the weights of each evaluation dimension based on contextual information associated with the accompanying medical service includes: increasing the weight of the communication clarity and service professionalism dimension when the contextual information includes the service type as accompanying medical treatment; and increasing the weight of the communication clarity and service attitude dimension when the contextual information includes the user tag as elderly.

6. The method according to claim 1, characterized in that, The preset evaluation dimensions include at least two of the following: service attitude, communication clarity, service professionalism, and user emotional satisfaction.

7. The method according to claim 6, characterized in that: When the preset multiple evaluation dimensions include communication clarity, the acoustic features include at least one of speech rate, volume, pitch, pause frequency and speech duration percentage, and are used to calculate the score of the communication clarity dimension. When the preset multiple evaluation dimensions include service attitude and / or user emotional satisfaction, the emotional characteristics include the duration of the companion and / or user's feelings of pleasure, impatience or dissatisfaction, and are used to calculate the score of the service attitude and / or user emotional satisfaction dimension. When the preset multiple evaluation dimensions include service attitude and / or service professionalism, the semantic features include the detection results of polite language, negative communication content, and key process confirmation, and are used to calculate the scores of the service attitude and / or service professionalism dimensions.

8. The method according to claim 1, characterized in that, The step of extracting semantic features representing the dialogue content based on the audio data includes: converting the audio data into text data using automatic speech recognition technology; and performing natural language processing on the text data to extract the semantic features.

9. The method according to claim 1, characterized in that, The service quality assessment result is a structured assessment report, which includes the overall score, scores for each assessment dimension, and targeted improvement suggestions generated based on the problem items identified in the score results.

10. A device for evaluating the quality of escort services based on audio analysis, characterized in that, include: The acquisition module is configured to acquire audio data representing the interaction between the escort and the user during the escort service. The feature extraction module is configured to process the audio data to extract acoustic features that reflect the physical properties of speech, identify the emotional state information of the dialogue participants as emotional features, and extract semantic features that characterize the content of the dialogue. The scoring module is configured to score the quality of the escort service based on the extracted multi-dimensional audio features and according to multiple preset evaluation dimensions, and obtain the score for each evaluation dimension. The weight adjustment module is configured to dynamically adjust the weights of each evaluation dimension based on context information associated with the accompanying medical service. The results generation module is configured to calculate a comprehensive service quality score based on the adjusted weights and scores of each evaluation dimension, and generate a service quality evaluation result based on the comprehensive service quality score.

Images