Method and apparatus for quality control based on service field data

Abstract

Embodiments of the present disclosure disclose a service site data-based quality control method and device. A specific implementation of the method comprises: receiving multimedia data uploaded by a terminal device in real time during a service process at a service site; the multimedia data comprises one or more of audio data, picture data and video data; determining, based on the multimedia data, to-be-prompted content during the service process; generating prompt information according to the to-be-prompted content; and sending the prompt information to the terminal device, so that the terminal device prompts a service personnel through one or more of a picture, text, sound and video. The implementation can monitor risks in the service process in real time and help the on-site service personnel improve service quality.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the fields of medical and health services and artificial intelligence technology, and specifically to a quality control method and apparatus based on service site data. Background Technology

[0002] The home-based medical services (such as in-home nursing services), housekeeping, and appliance repair services are experiencing rapid growth. These services rely on the private setting of a home, presenting common challenges such as high performance risks, difficulty in process monitoring, and difficulty in defining liability. Large internet service platforms, due to their large order volumes, dispersed locations, and high staff turnover, are highly susceptible to public opinion crises and brand / business crises caused by substandard home services and service disputes, making a real-time and efficient performance monitoring system urgently needed.

[0003] Currently, the industry generally adopts a post-event supervision model that involves recording service personnel's terminal audio and manually sampling or reviewing recordings after the service is completed, or reviewing complaints. This model is insufficient to meet the needs of large platforms for comprehensive, real-time, and refined risk control and quality supervision of on-site services. There is an urgent need for new intelligent on-site service process supervision technology solutions and tools to help on-site service personnel improve service quality. Summary of the Invention

[0004] Embodiments of this disclosure present a quality control method and apparatus based on service site data.

[0005] In a first aspect, embodiments of this disclosure provide a quality control method based on service site data, comprising: receiving multimedia data uploaded in real time by a terminal device during the service process at a service site; the multimedia data including one or more of audio data, image data, and video data; determining content to be prompted during the service process based on the multimedia data; generating prompt information according to the content to be prompted; and sending the prompt information to the terminal device, so that the terminal device can issue prompts to service personnel through one or more of images, text, sound, and video.

[0006] In some embodiments, determining the content to be prompted during the service process based on the multimedia data includes: identifying the multimedia data to obtain an identification result; obtaining a pre-created standard operating procedure for the service site, wherein the standard operating procedure includes: standard actions and quality control points for the standard actions; inputting the identification result and preset action prompts into a large language model to determine the content to be prompted, wherein the action prompts are used to guide the large language model to determine the next standard action and precautions as the content to be prompted by analyzing the identification result, so that the action performed by the service provider with reference to the content to be prompted meets the requirements of the quality control points.

[0007] In some embodiments, generating prompt information based on the content to be prompted includes: generating multimedia demonstration data of the standard action based on the content to be prompted, wherein the multimedia demonstration data includes one or more of audio data, image data, and video data.

[0008] In some embodiments, determining the content to be prompted during the service process based on the multimedia data includes: identifying the multimedia data to obtain an identification result; inputting the identification result and preset operation specification prompts into a large language model to determine whether there is a risk in the service process, wherein the operation specification prompts are used to guide the large language model to analyze whether the service provider's operation conforms to standard operation specifications; and in response to non-compliance with standard operation specifications, determining the content to be prompted based on the non-compliance operation and the corresponding standard operation.

[0009] In some embodiments, generating prompt information based on the content to be prompted includes: generating multimedia error correction data to correct operations that do not conform to standard operating procedures, so that the service provider can correct operations that do not conform to standard operating procedures, wherein the multimedia error correction data includes one or more of audio data, image data, and video data.

[0010] In some embodiments, the method further includes: in response to the end of the service, identifying the complete multimedia data of the service process to obtain an identification result; performing a compliance check on a standard operating procedure based on the identification result to obtain an inspection result, wherein the standard operating procedure includes: standard actions and quality control points for the standard actions; sending a service quality questionnaire to the terminal of the service recipient; and in response to receiving feedback information from the service recipient regarding the service quality questionnaire, generating a quality control report based on the feedback information and the inspection result.

[0011] In some embodiments, the method further includes: sending the quality control report to the terminal device; archiving the quality control report in response to receiving a quality control report confirmation message from the terminal device; reviewing the quality control report based on an artificial intelligence model in response to receiving an appeal request from the terminal device; and archiving the quality control report in response to the review result being consistent with the quality control report.

[0012] In some embodiments, the method further includes: calibrating the discrepancies between the review results and the quality control report in response to discrepancies; regenerating the quality control report based on the calibrated discrepancies; and archiving the regenerated quality control report. In response to the quality control report meeting predetermined conditions, optimizing standard operating procedures based on the quality control report.

[0013] In some embodiments, the method further includes: in response to the end of the service, identifying the complete multimedia data of the service process to obtain an identification result; generating guidance information based on the identification result and the target of the service item; and sending the guidance information to the terminal device and / or the terminal of the service recipient.

[0014] Secondly, embodiments of this disclosure provide a quality control device based on service site data, comprising: a receiving unit configured to receive multimedia data uploaded in real time by a terminal device during the service process at the service site; the multimedia data including one or more of audio data, image data, and video data; a determining unit configured to determine, based on the multimedia data, content to be prompted during the service process; a generating unit configured to generate prompt information according to the content to be prompted; and a prompting unit configured to send the prompt information to the terminal device, so that the terminal device can issue prompts to service personnel through one or more of images, text, sound, and video.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors perform the method as described in any one of the first aspects.

[0016] Fourthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of the first aspects.

[0017] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0018] The quality control method and apparatus based on service site data provided in the embodiments of this disclosure can detect risks in the service process in real time by collecting voice and images during the service process, thereby reducing labor costs and helping on-site service personnel improve service quality.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied; Figure 2 This is a flowchart of an embodiment of a quality control method based on service site data according to the present disclosure; Figure 3 This is a flowchart of yet another embodiment of the quality control method based on service site data according to the present disclosure; Figures 4a-4e This is a schematic diagram illustrating an application scenario of the quality control method based on service site data according to this disclosure; Figure 5 This is a schematic diagram of a structure of an embodiment of a quality control device based on service site data according to the present disclosure; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0021] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Figure 1 An exemplary system architecture is shown that can be applied to embodiments of the quality control method or device based on service site data disclosed herein, using a nurse home visit service scenario as an example, but it can also be applied to other home visit service scenarios.

[0024] like Figure 1 As shown, the system architecture for applying a quality control method based on service site data includes: 1. Infrastructure Layer It provides computing resources (GPU / CPU), network transmission and cloud platform services to provide computing power and basic environment support for upper-layer AI models, real-time processing and data storage.

[0025] 2. Data Layer

[0026] Business database: Stores core business data such as SOPs (Standard Operating Procedures) and service order records.

[0027] Quality control database: Stores AI recognition results, quality control scores, appeal records, and modification logs.

[0028] Multimedia storage: Stores original materials such as on-site audio, recordings, and service site photos.

[0029] Knowledge Base: Accumulates high-quality service cases, educational materials, and SOP templates for training and AI-assisted SOP generation.

[0030] 3. Core Service Layer

[0031] a. Real-time processing service: 1) Real-time ASR (Speech Recognition): Converts on-site audio recordings into text in real time.

[0032] 2) Real-time visual model analysis: Perform compliance identification on on-site images.

[0033] 3) Real-time LLM (Large Language Model) risk monitoring: Use LLM to provide risk warnings for real-time text.

[0034] b. Offline / Batch Processing Service: 1) Full-process ASR (Automatic Speech Recognition): After the service is completed, the entire recording is re-recognized.

[0035] 2) LLM full-process SOP check: Perform compliance verification on the complete service process.

[0036] 3) AI-automated review: Processing service personnel's appeals.

[0037] 4) SOP generation service: LLM assists in generating new service SOP templates.

[0038] c. Calculation Engine: Scoring engine: Automatically calculates the total quality control score based on SOP compliance.

[0039] Case study engine: Analyzes and preserves high-quality service cases.

[0040] 4. Application Layer (Business Function Modules)

[0041] SOP Management System: Enables managers to develop SOP processes and configure risk implementation items.

[0042] Service Execution System: The core functions of the nurse's app include on-site audio recording, audio editing, and image capture.

[0043] Quality control and analysis system: Enables real-time monitoring, provides prompts for the service process, conducts full-process quality control, calculates scores, analyzes high-quality cases, accumulates knowledge, and disseminates educational information.

[0044] User interaction system: allows service personnel to view quality control results, file appeals and receive confirmations, and also supports manual calibration.

[0045] 5. User and Terminal Layer

[0046] Nurses: Perform services, collect data, receive prompts and quality control results, and file appeals through the nurse app.

[0047] Administrators: Configure SOPs, view quality control data, and handle manual calibrations through the back-end management system.

[0048] Back-end management system: Provides management personnel with a platform management portal.

[0049] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a quality control method based on service site data according to the present disclosure. This quality control method based on service site data includes the following steps: Step 201: Receive multimedia data uploaded in real time by the terminal device during the service process at the service site.

[0050] In this embodiment, multimedia data includes one or more of audio data, image data, and video data. When service personnel perform on-site services through a service personnel-endorsed App (terminal device), the App activates on-site audio and image / video capture functions. During the service process, the captured multimedia data is uploaded to the service platform server in real time at preset time intervals (e.g., every minute) or at key service nodes. Simultaneously with audio data collection, service personnel take on-site service photos and upload them to the server when performing key operations (e.g., disinfection, medication administration, puncture, etc.). Optionally, video data from the service process at the service site can be uploaded in real time.

[0051] Step 202: Based on multimedia data, determine the content to be prompted during the service process.

[0052] In this embodiment, after receiving the audio data uploaded by the terminal device, the server invokes a real-time ASR (Automatic Speech Recognition) service to perform speech recognition on the audio data, converting the dialogue between the service personnel and the service recipient (user) into structured text to obtain the speech recognition result. The recognition result may include information such as the dialogue content, the speaker's role (nurse / user), and key timestamps.

[0053] After receiving service images or video data uploaded by the terminal device, the server invokes the visual model analysis service to perform image recognition on the images or video data. This recognition identifies information such as the service scenario, operational actions, consumable usage, and user status, yielding the image recognition results. For example, it can identify whether nurses are wearing masks and gloves correctly, using disposable consumables correctly, and performing skin disinfection according to standard procedures.

[0054] Based on the recognition results of multimedia data, a pre-trained large language model can be used, combined with standard service specifications and risk control rules corresponding to the service scenario, to conduct a comprehensive analysis of service behavior, on-site environment, and personnel communication content. This analysis ultimately determines the content to be prompted during the service process. The content to be prompted is divided into two main categories: risk warnings and operational guidance. The specific determination method is as follows: (a) Determine the content of the risk warning generated. For service scenarios with potential safety hazards, service violations, inappropriate communication, or procedural anomalies, the system generates risk warnings. For example, in scenarios such as nurses providing home-based infusions, dressing changes, catheter care, and chronic disease follow-ups, the system analyzes on-site video and image data to identify operational risks such as nurses failing to strictly adhere to hand hygiene, not wearing medical masks and gloves, not properly sterilizing medical equipment, haphazardly placing medical consumables, damaged or contaminated infusion sets, and preparing for procedures without verifying patient identity and medical orders. Audio data identifies communication risks such as nurses failing to truthfully disclose nursing risks, using misleading language regarding treatment outcomes, giving perfunctory responses to patient questions, and causing emotional conflict between doctors and patients. The system also identifies on-site safety hazards in the patient's home environment, such as clutter obstructing the operating area, slippery floors, open flames near medical equipment, and family members touching sterile instruments without authorization. All identified medical violations, operational hazards, communication risks, and environmental risks generate corresponding risk warning content.

[0055] (ii) Determine the content of the generated operation guide.

[0056] For scenarios where service personnel operate improperly, omissions in procedures, errors in operational steps, or substandard service standards occur without posing potential risks, the system generates operational guidance content. For example, in a nurse's home visit for intravenous puncture, video and image data are used to identify instances of improper skin disinfection, incorrect patient positioning, and incorrect tourniquet application. In a wound dressing change scenario, incorrect dressing change sequence, omissions in wound cleaning steps, and improper dressing fixation are identified. In a home visit for chronic disease patients, failure to measure vital signs such as blood pressure, blood glucose, and blood oxygen saturation according to standard procedures, incomplete recording of changes in the patient's condition, and failure to provide standardized guidance on medication and rehabilitation care are identified. In a catheter placement care scenario, non-standard procedures for catheter fixation, flushing, and sealing are identified. For these behaviors that do not conform to medical and nursing standards or service procedures, standardized medical operation guidance content is generated to be provided.

[0057] Optionally, the server performs real-time risk assessment of the service process based on speech recognition and / or image recognition results, combined with preset service SOPs and risk execution items, for example: 1. When the voice recognition results contain high-risk dialogue content such as user questioning of operations, service personnel making illegal promises, or users expressing physical discomfort, it is determined that there is a communication or operational risk in the service process; 2. When the image recognition results detect abnormalities such as service personnel not wearing protective equipment as required, operating procedures not conforming to SOPs, or incorrect use of consumables, it is determined that there are operational or compliance risks in the service process; 3. If both the speech recognition result and the image recognition result have risk characteristics, the confidence level of the risk assessment can be further improved.

[0058] Step 203: Generate prompt information based on the content to be prompted.

[0059] In this embodiment, based on the identified content to be prompted, the risk level and operation scenario are distinguished, and a unique home care prompt template is matched. Combined with the specific on-site problem and the type of care service, accurate, professional, and directly applicable personalized prompt information is generated. The prompt information supports multiple forms such as text, voice, graphic guidance, and standardized operation short videos, and is adapted to different types of content to be prompted.

[0060] For example, regarding the high-risk warning message "Patient identity and medical orders not verified, posing a medical risk during intravenous infusion," the system generates a professional text prompt: "Patient identity and medical orders have not been verified. Intravenous infusion is strictly prohibited. Please complete the double verification immediately and continue the operation only after confirming that the information is consistent." This is accompanied by a serious medical risk warning voice message. For operational guidance regarding "inadequate wound disinfection or non-standard procedures," the system generates illustrated guidance prompts, indicating the standard disinfection range, disinfection methods, and operational steps, and provides a short video of the official standardized dressing change procedure for nurses to refer to and correct. For service oversights such as "failure to inform patients of home medication precautions," the system generates standardized voice and text prompts, clearly specifying the core information that needs to be informed, such as medication contraindications and follow-up appointment times.

[0061] Step 204: Send the prompt information to the terminal device so that the terminal device can issue a prompt to the service personnel through one or more of the following: images, text, sound, and video.

[0062] In this embodiment, various generated prompts are sent to the nurse's handheld terminal device in real time. The terminal device uses one or more combinations of images, text, sound, and video to send a prominent reminder to the nurse on site, based on the prompt type and risk level, ensuring that the nurse can perceive the problem and complete the rectification as soon as possible.

[0063] For example, high-level risk warnings involving medical safety, infection risk, and operational violations are presented using a combination of "red text pop-up forced to the top + high-decibel voice broadcast + risk warning image" to forcibly interrupt the current operation and remind nurses to pause their work and investigate risks. Guidance prompts for non-standard routine nursing operations or omissions in procedures are presented using "text pop-up + standardized operation graphics / short video" to facilitate nurses' review and rectification without affecting the progress of normal nursing services. Minor oversights in service details are only reminded through gentle voice broadcasts, which are concise, efficient, and suitable for home service scenarios.

[0064] Based on real-time prompts from the terminal device, nurses can promptly terminate high-risk and unethical operations, correct non-standard nursing actions, complete missing service procedures, and avoid potential safety hazards in the home environment. This enables dynamic rectification of problems in home nursing services and ensures that the entire nursing operation is compliant, safe, and standardized.

[0065] Optionally, in response to a determination that a risk exists in the service process, the server generates and outputs an alarm message. The methods for pushing alarm messages include, but are not limited to, the following: 1. Push real-time pop-up alerts to the service personnel's app to remind them that there are risks in the current service and that they need to correct their operations or adjust their communication methods in a timely manner; 2. Push alarm notifications to the platform administrators in the backend, and synchronize the risk level, risk type, occurrence time and related audio and image data to facilitate real-time intervention by the platform administrators; 3. For high-risk situations (such as severe discomfort experienced by users or serious violations of regulations by nurses), an emergency intervention process can be triggered, automatically generating an alarm work order and notifying the platform's quality control personnel or emergency response team.

[0066] The method provided in the above embodiments of this disclosure combines voice recognition and image recognition to achieve multimodal real-time risk assessment and alarm in the service process. It changes the traditional post-event manual review mode, effectively solves the problems of lagging control, low efficiency and high labor costs, improves the accuracy and coverage of risk identification, can intervene in violations in a timely manner during the service process, standardize service processes, reduce service disputes, and strengthen the platform's risk control capabilities and service standardization level.

[0067] Figure 4a This is a flowchart using home nursing care services as an example. The specific process is as follows: 1. Data Acquisition Layer During a nurse's home visit, data is collected through the nurse's mobile app's data collection module. The app continuously collects audio data during the service process. The app also captures images of the service location (such as the operational scene, consumables, and nurse's attire). The collected audio and images are initially stored locally on the app and then uploaded to the platform server at minute intervals.

[0068] 2. Real-time processing layer

[0069] The platform server processes the uploaded data in three parallel streams, ultimately fusing the data to output a risk assessment result. Uploaded audio data is sent to the real-time audio data service, while uploaded image data is sent to the on-site image service and processed by the visual model analysis service.

[0070] The model performs service operation standard identification and outputs a judgment result on "whether the service operation is standardized" (e.g., nurse attire, use of consumables, and whether the operation steps conform to SOP). ASR transcribed text and visual model analysis results, combined with Prompt project configuration (preset risk warning words), are input into the large language model risk monitoring module. The model integrates multimodal information to perform real-time risk monitoring and identify potential risks in the service process (e.g., improper communication, operational violations, user risk statements, etc.).

[0071] 3. Output / Result Layer: Risk Warning and Push Notification

[0072] The results of large language models and visual analysis jointly trigger real-time quality control early warning pushes.

[0073] The system generates potential risk warning information and pushes it synchronously to: Nurse-side APP: Real-time pop-up alerts prompt nurses to correct operations or adjust communication methods.

[0074] Management backend: Allows platform quality control personnel to intervene in real time and synchronize risk details and related data.

[0075] In some optional implementations of this embodiment, determining the content to be prompted during the service process based on multimedia data includes: recognizing the multimedia data to obtain the recognition result; obtaining a pre-created standard operating procedure for the service site, wherein the standard operating procedure includes: standard actions and quality control points for the standard actions; inputting the recognition result and preset action prompt words into a large language model to determine the content to be prompted, wherein the action prompt words are used to guide the large language model to determine the next standard action and precautions as the content to be prompted by analyzing the recognition result, so that the action performed by the service provider in accordance with the content to be prompted meets the requirements of the quality control points.

[0076] Operations personnel complete the initial configuration of quality control rules on the management side, supporting two modes: 1. Manual configuration: Operations personnel can customize the standard operating procedure (SOP) for the home delivery service project according to business needs and configure risk execution items.

[0077] 2. AI Intelligent Generation: Automatically generate standard operating procedures (SOPs) for projects using AI capabilities, improving configuration efficiency.

[0078] Managers can create standard operating procedures (SOPs) for different service items through the SOP library management interface, such as "Port maintenance SOP" and "PICC placement care SOP". Each SOP is associated with preset standard actions and corresponding quality control points, such as "verify name", "confirm age", "upload red nursing kit", and "wear nurse uniform". Among them, voice-based quality control points are configured with corresponding quality control point prompts. For example, the prompt for the "verify name" action can be set to "analyze the dialogue content to determine whether the service provider has verified the name with the user, and whether there are any cases of non-verification, name mismatch, or user non-confirmation".

[0079] The terminal device continuously collects multimedia data from the site in real time, including one or more of the following: operation videos, captured images, and audio recordings of doctor-patient communication, and uploads them to the cloud backend in real time. After acquiring the multimedia data, the cloud backend performs multi-dimensional analysis of the data using image recognition, behavior recognition, and speech recognition algorithms, and outputs structured recognition results. The recognition results accurately represent the nurse's current operational status, operational actions, process progress, and on-site compliance.

[0080] Taking a home-based wound dressing change scenario as an example: the system identifies the video and image data from the scene, determining that the nurse has completed wound cleaning but has not yet performed skin disinfection, and is not wearing sterile gloves; the work surface is not sterile. Audio recognition confirms that the nurse has not yet informed the patient of dressing change precautions and post-operative care points. The final output is a structured recognition result, including key information such as currently performed actions, unperformed actions, non-standard actions, the state of the scene environment, and the progress of each process node.

[0081] The recognition results and preset action prompts are input into the large language model, which then combines standard operating procedures and corresponding quality control points to intelligently analyze and output the final prompt content.

[0082] The custom action prompts serve to guide the large language model to combine the current on-site recognition results with the standard actions and quality control points of the standard operating procedures, identify any omissions or non-standard items in the current operation, and accurately output the standard actions, detailed operating methods, and compliance precautions that the service personnel need to perform next, ensuring that the actions performed by the service personnel in the future fully meet the quality control point specifications.

[0083] Example input content: 1. Recognition result: The nurse has completed wound cleaning, but did not perform hand hygiene, wear sterile gloves, disinfect the skin, or inform the patient of nursing precautions. The sterile items on the operating table are not placed in accordance with regulations. 2. Action prompt: Based on the current on-site recognition result of the home dressing change, please compare it with the standard operating procedures and quality control points for home dressing change, and output the standard actions, operational details, and compliance precautions that the nurse must perform next to ensure that all operations meet the nursing quality control requirements.

[0084] After inference and analysis, the large language model outputs structured prompts: First, immediately pause the dressing change procedure, strictly follow the seven-step handwashing method to complete hand hygiene, and wear sterile medical gloves and masks properly to meet the quality control requirements for aseptic operation; Second, tidy up the work surface, separating sterile consumables from general waste to avoid the risk of cross-infection; Third, disinfect the wound and surrounding skin in an inward-outward order, extending the disinfection area 3-5cm beyond the wound to meet disinfection quality control standards; Fourth, after disinfection, allow the wound to dry completely, and do not repeatedly wipe the wound; Fifth, after the procedure, inform the patient and their family in detail about wound care contraindications, precautions against contact with water, and prompts for follow-up visits in case of abnormalities.

[0085] The next standard actions, operational details, and compliance precautions outlined in the above output constitute the final reminders for this service process.

[0086] Optionally, the server obtains the SOP corresponding to the current service item and the associated quality control point prompts, and inputs the recognition results and quality control point prompts into the large language model. Guided by the prompts, the model verifies the recognition results one by one to determine whether the service process meets the requirements of each quality control point, such as whether name verification, age confirmation, and pre-service risk assessment have been completed. If the verification finds that quality control points have not been executed as required, dialogue is not standardized, or key confirmation steps have not been completed, then the service process is deemed to have risks.

[0087] Based on the quality control verification results output by the large language model, the server marks the corresponding quality control points for service orders with risks and generates alarm information, which is then pushed to service personnel and platform administrators for timely intervention and subsequent verification.

[0088] In this embodiment, the quality control requirements of the SOP are transformed into prompts that can be recognized by a large language model. Combined with the speech recognition results, the quality control points are automatically verified during the service process. This ensures the consistency between the quality control logic and the preset service standards, and also achieves intelligent matching of dialogue content and quality control requirements based on the large language model. Compared with the traditional manual order-by-order verification, this significantly improves quality control efficiency, reduces labor costs, and enables accurate verification of quality control points and rapid identification of risk points. It ensures the standardized execution of the service process and effectively avoids service risks and platform disputes caused by omissions or non-standard operations of quality control points.

[0089] In some optional implementations of this embodiment, generating prompt information based on the content to be prompted includes: generating multimedia demonstration data of standard actions based on the content to be prompted, wherein the multimedia demonstration data includes one or more of audio data, image data, and video data.

[0090] Based on the content to be prompted obtained from the analysis, the built-in nursing standard action material library is accurately matched to automatically generate corresponding multimedia demonstration data. The multimedia demonstration data includes one or more combinations of audio data, image data, and video data. All demonstration data strictly conforms to the quality control point requirements of the standard operating procedure and corresponds one-to-one with the content to be prompted, and is accurately adapted.

[0091] The resource library pre-stores standardized demonstration materials for various in-home nursing scenarios, covering standard step-by-step diagrams, high-definition demonstration videos, and standardized audio explanations for each operational step. All materials have undergone medical and nursing quality control calibration and fully comply with clinical nursing standards. The system adaptively combines and generates corresponding multimedia demonstration data based on the type and content of the prompts to be provided. Specific scenario examples are as follows: If the prompt message is "Wound disinfection operation is not standardized, the disinfection area is insufficient, or the operation sequence is incorrect. Disinfection should be carried out from the inside out, and the disinfection area should extend 3-5cm beyond the wound surface," the system will generate corresponding multimedia demonstration data: including high-definition images of standard wound disinfection steps, a short video of the complete disinfection operation, and synchronized audio explanations. The images will indicate the standard disinfection area and operating techniques; the video will dynamically demonstrate the complete standardized disinfection process; and the audio will synchronously explain key points of disinfection quality control and contraindications.

[0092] In some optional implementations of this embodiment, determining the content to be prompted during the service process based on multimedia data includes: identifying the multimedia data to obtain the identification result; inputting the identification result and preset operation specification prompts into a large language model to determine whether there is a risk in the service process, wherein the operation specification prompts are used to guide the large language model to analyze whether the service provider's operation conforms to the standard operation specification; and in response to non-compliance with the standard operation specification, determining the content to be prompted based on the non-compliance operation and the corresponding standard operation.

[0093] Pre-configured operation guidelines and prompts are adapted to various home care scenarios. These prompts guide the large language model to compare the operation guidelines with clinical nursing standards and analyze, item by item, whether the service provider's on-site operation behavior is compliant and whether there are any missing procedures or operational risks.

[0094] In this embodiment, the on-site identification results obtained above, along with the preset nursing operation standard prompts, are input into the large language model. The specific guidance logic of the operation standard prompts is as follows: based on the input on-site identification results of the nurse's home care visit, the model compares the nurse's current operation with the national standard operating procedures and quality control specifications, judging whether each operation is compliant, accurately identifying non-standard operations, violations, and operations with safety risks, and outputting a clear compliance judgment result.

[0095] The large language model performs intelligent reasoning and analysis based on the input content, and finally determines whether there are any behaviors that do not conform to standard operating procedures or whether there are any medical safety risks in the nursing service process.

[0096] Taking wound dressing as an example again: Based on the recognition results and in accordance with the standard dressing procedure, the large language model determines that there are several non-standard operating procedures, such as failure to perform hand hygiene, failure to wear sterile gloves, and missing skin disinfection procedures, which pose a risk of wound infection and constitute a violation of the procedure.

[0097] In some optional implementations of this embodiment, generating prompt information based on the content to be prompted includes: generating multimedia error correction data to correct operations that do not conform to standard operating procedures, so that the service provider can correct operations that do not conform to standard operating procedures. The multimedia error correction data includes one or more of audio data, image data, and video data.

[0098] Based on the generated prompt content, a pre-built standardized nursing error correction material library is retrieved, and multimedia error correction data for correcting non-standard operations is intelligently matched and generated. This error correction data includes one or more combinations of audio data, image data, and video data. All materials accurately correspond to the rectification needs of this violation and fully comply with the nursing standard operating procedures.

[0099] The resource library contains standardized error correction materials for various in-home nursing scenarios, including comparative diagrams of violations, real-life photos of standard rectification steps, short videos of standardized operation tutorials, and step-by-step error correction audio explanations. It can adaptively match and generate customized error correction data according to different violation scenarios. Specific scenario examples are as follows: For portal intravenous infusion scenarios, the prompt message is: "Failure to perform the three checks and seven rights, failure to inquire about allergy history, and direct commencement of puncture preparation poses a medication safety risk." The system generates corresponding multimedia error correction data: including infographics and images correcting errors in the infusion verification standard process, a demonstration video of compliant puncture preparation operations, and audio broadcasts of infusion safety verification precautions, providing comprehensive guidance to nurses to correct violations. In some optional implementations of this embodiment, the method further includes: in response to service completion, identifying the complete multimedia data of the service process to obtain identification results; generating guidance information based on the identification results and the goals of the service items; and sending the guidance information to the terminal device and / or the terminal of the service recipient.

[0100] This embodiment takes an on-site service scenario (such as housekeeping, appliance repair, and in-home nursing) as an example to provide a method for generating and pushing post-service guidance information, which is executed by the server. The specific steps are as follows: Once the service personnel submit a service completion instruction via the terminal app, the server automatically retrieves the complete multimedia data collected and stored during the service process, calls the full-process ASR speech recognition service and image recognition service, and performs a complete transcription of the multimedia to obtain recognition results containing information such as the entire service process dialogue content, key interaction nodes, user feedback, and service personnel explanations.

[0101] Based on the preset goals of this service project (such as "home care precautions after surgery," "key points for use and maintenance after appliance repair," and "suggestions for maintaining the home environment after housekeeping"), the server combines information such as the user's special circumstances, service personnel's operational details, and user questions recorded in the audio recognition results to generate targeted guidance information through a large language model. For example, based on the service goal of "postoperative care" and the user's mention of skin sensitivity in the recording, personalized home care guidance including wound care, medication reminders, and handling of abnormal situations can be generated.

[0102] The server will push the generated guidance information to the service personnel's app and / or the service recipient's terminal device (such as the user's mobile app, SMS, etc.). The guidance information pushed to the service personnel's terminal can serve as supplementary content for the service quality closed loop and as a reference for subsequent service follow-ups. The guidance information pushed to the service recipient's terminal is presented in a user-friendly, structured format, including key operating steps, precautions, contact information, etc., for the user's convenience in subsequent reference and execution.

[0103] In this embodiment, after the service is completed, personalized guidance information is generated and pushed by performing voice recognition on the complete recording data and combining it with the service project objectives. This extends the door-to-door service from "single performance" to "full-cycle service loop". On the one hand, the guidance information generated based on the complete recording content is more in line with the user's actual situation, solving the problem that traditional general guidance content is not targeted and easily overlooks the user's special needs, effectively improving the user service experience and satisfaction. On the other hand, through two-way push between the service personnel end and the user end, the integrity and professionalism of service performance are strengthened. At the same time, the service process and user needs data are accumulated, providing a reliable basis for subsequent service optimization, user follow-up and dispute resolution, further improving service quality control and user stickiness.

[0104] Further reference Figure 3 This illustrates a flow 300 of another embodiment of a quality control method based on service site data. The flow 300 of this quality control method based on service site data includes the following steps: Step 301: Receive multimedia data uploaded in real time by the terminal device during the service process at the service site; Step 302: Based on multimedia data, determine the content to be prompted during the service process; Step 303: Generate prompt information based on the content to be prompted; Step 304: Send the prompt information to the terminal device so that the terminal device can send a prompt to the service personnel through one or more of the following: images, text, sound, and video.

[0105] Steps 301-304 are basically the same as steps 201-204, so they will not be described again.

[0106] Step 305: In response to the end of the service, identify the complete multimedia data of the service process and obtain the identification result; In this embodiment, after the service personnel submit a service completion instruction through the terminal App, the server automatically retrieves the complete multimedia data collected and stored during the service process, calls the full-process ASR speech recognition service and image recognition service, and performs a complete transcription of the multimedia data to obtain recognition results containing information such as the entire service process dialogue content, key interaction nodes, service personnel operation instructions, and user communication records.

[0107] Step 306: Based on the identification results, perform a compliance check on the standard operating procedures to obtain the check results.

[0108] In this embodiment, the server obtains the standard operating procedure (SOP) corresponding to this service project. This SOP includes preset standard actions and quality control points for each action (such as "verifying user information before service", "informing users of risk matters", and "explaining precautions after service"). The audio recognition results are input into the large language model, and each quality control point is checked item by item according to the inspection rules: it is determined whether all standard actions were completed according to the SOP requirements during the service process, and whether the compliance requirements of each quality control point are met. Finally, the SOP compliance check result is output, including compliant items, non-compliant items, missing items, and corresponding timestamps.

[0109] Step 307: Send a service quality questionnaire to the terminal of the service recipient.

[0110] In this embodiment, after the service is completed, the platform automatically sends a service quality questionnaire to the service recipient's terminal device (such as a user's mobile app, SMS link, etc.). The questionnaire can include multi-dimensional evaluation questions on service personnel professionalism, service attitude, operational standardization, communication experience, and problem-solving status. Users can choose to rate the service or fill in written feedback, comprehensively collecting users' subjective experience information on this service.

[0111] After the service recipient completes the questionnaire and submits feedback, the platform receives and stores the user's questionnaire feedback results in a structured manner, including scores for each dimension, textual evaluation content, improvement suggestions or complaints made by the user, etc.

[0112] The platform obtains the SOP compliance check results for this service (based on operational compliance and quality control point completion status obtained from audio recordings and image recognition), and integrates and analyzes them with user feedback: For compliance items that are consistent with user feedback and SOP check results, service quality is further confirmed; for negative evaluations, complaints, or dissatisfaction from users, cross-validation is performed in conjunction with the SOP check results to supplement and mark problems in the service or risk points of concern to users; by combining the SOP compliance score and user feedback score, a complete quality control report is generated, which includes service process compliance, user experience evaluation, problem analysis, and comprehensive score. At the same time, it can automatically generate targeted service improvement suggestions based on user feedback.

[0113] Step 308: In response to receiving feedback from the service recipient regarding the service quality questionnaire, generate a quality control report based on the feedback and inspection results.

[0114] In this embodiment, the server automatically generates a structured quality control report based on the SOP compliance check results and the service recipient's feedback on the service quality questionnaire. The report may include basic information about the service provided, SOP compliance rate statistics, detailed compliance / non-compliance information, explanations of quality control point completion, and scoring / deduction criteria. In some scenarios, it can also automatically generate rectification suggestions based on non-compliance items. The generated quality control report can be stored in the platform's quality control database and can also be pushed to service personnel and the platform management backend for viewing.

[0115] The quality control report supports quick location of target orders across multiple dimensions, including performance order number, order number, service personnel name, case classification, appeal type, review role, review result, city, service item, service personnel ID, quality control score range, service completion time, anomaly tags, nurse's full-time / part-time status, and service tags. It also provides "reset" and "query" functions, allowing users to clear or apply filters with a single click.

[0116] The quality control report, presented in a structured format, comprehensively outlines the compliance inspection results and scoring details for the entire service order process. Its core components include the following: 1. Overall Score and Explanation The top of the report displays the total quality control score for this service (90 points in the example), and notes the limitations of the applicable scenarios for AI quality control (such as better results in single-person, single-service scenarios, while multi-person, multi-project scenarios are still under exploration), indicating that this quality control was based solely on audio and image data from the service process.

[0117] 2. Detailed quality control process in stages

[0118] The report is divided into multiple stages according to the service process (such as "Pre-arrival call recording quality control" and "Post-arrival service recording quality control"), with each stage listed in tabular form: Standard Action ID / Name / Description: The standard operations preset in this service SOP (such as "self-introduction", "service information confirmation", "entry actions", "wash hands before service", etc.), which clearly define the execution requirements and script specifications for this action.

[0119] Quality Control Point ID / Quality Control Description: The key points for checking the corresponding standard actions, such as "checking whether the self-introduction has been completed", "confirming the user's service information", "whether shoes have been changed and items have been placed according to regulations when entering the home", etc. Some quality control points are accompanied by sample scripts for reference.

[0120] Quality control results: The AI ​​model's verification conclusion for this quality control point (e.g., "passed"), and the corresponding deduction (in the example, the initial deduction is 0).

[0121] Appeal / Review Status: Shows whether there are any appeals or reviews at this quality control point.

[0122] 3. Quality control rule configuration

[0123] The report's quality control logic is derived from the standard actions and quality control points configured in the backend: Standard Action Configuration: Managers can add / edit standard actions for service items (such as "Traditional Chinese Medicine - Preparation of Service Materials"), and specify the action description and execution requirements.

[0124] Quality control point configuration: Configure corresponding AI quality control rules for each standard action, including: Score: The weight of this quality control point in the total score (e.g., 10 points). Quality control logic: Select the "AI Quality Control" mode, and the large language model will automatically check the quality. Real-time quality control: Configure whether this quality control point needs to be triggered in real time during the service process. AI Quality Control Prompt: Verification prompts provided for large models (such as verification phrases like "The service provider informs the user that they will prepare the corresponding service items"). Deduction criteria: Define which results (such as "not mentioned" or "not completed") will trigger point deductions. This solution automates and standardizes post-service quality control by completing compliance checks and generating quality control reports after the service is completed. On one hand, relying on AI recognition and large-scale model verification replaces the traditional method of manually reviewing each order's recordings, significantly reducing labor costs and improving quality inspection efficiency and coverage, thus adapting to the quality inspection needs of large platforms with massive service orders. On the other hand, the item-by-item verification based on SOP quality control points ensures the consistency and objectivity of quality inspection standards, avoiding subjective bias and omissions inherent in manual review. Simultaneously, the generated structured quality control report clearly presents the compliance status of the service process, providing reliable data support for service quality assessment, personnel evaluation, dispute resolution, and service process optimization, effectively promoting the standardization and normalization of on-site service management.

[0125] In some optional implementations of this embodiment, the method further includes: sending a quality control report to a terminal device; and archiving the quality control report in response to receiving a quality control report confirmation message from the terminal device.

[0126] In some optional implementations of this embodiment, the method includes: in response to receiving an appeal request from a terminal device, reviewing the quality control report based on an artificial intelligence model; and in response to the review result being consistent with the quality control report, archiving the quality control report.

[0127] like Figure 4b and 4c As shown, taking the AI ​​quality control system for nurse home care services as an example, this paper provides a method for pushing, confirming, appealing, reviewing, and archiving quality control reports. This method is executed collaboratively by the platform server and the nurse's terminal device. The specific steps are as follows: After the service is completed, the platform generates a quality control report that includes SOP compliance check results, user feedback, and a comprehensive score, and pushes it to the nurse's app in real time. After viewing the quality control report on their terminal device, the nurse can choose to confirm the report content. Upon receiving the nurse's confirmation message, the platform marks the quality control report as undisputed and archives it in the quality control database, thus completing the closed-loop quality control process.

[0128] If a nurse disagrees with the quality control report, they can initiate an appeal request on the terminal device. After receiving the appeal request, the platform will activate the AI ​​system's automatic review mechanism. Based on the original service recordings, image recognition data, user feedback information, and SOP quality control rules, the AI ​​system will conduct multiple rounds of intelligent review of the disputed items in the quality control report, re-verifying the compliance of the service process and the risk assessment results.

[0129] After the AI ​​review is completed, the platform compares the review results with the original quality control report: if the review results are consistent with the conclusion of the original report, it means that the nurse's appeal has no valid basis, and the platform directly archives the original quality control report and maintains the original quality control conclusion; if there are differences between the review results and the original report, the disputed content is automatically marked and transferred to the manual confirmation stage, where quality control personnel will manually calibrate it.

[0130] Quality control personnel manually verify and calibrate any discrepancies in the marked items, revise the quality control report based on the verification results, and recalculate the service score. After calibration, the system archives the revised quality control report and analyzes high-quality service cases, storing them in the knowledge base for subsequent nurse training and service process optimization.

[0131] This solution establishes a complete closed-loop process: "AI quality control - nurse confirmation / appeal - AI review - manual calibration - report archiving." This effectively solves the problems of traditional quality control reports being unable to be appealed by service personnel and lacking a verification mechanism. On one hand, the push and confirmation mechanism for quality control reports enhances service personnel's right to know and participate in quality control results, reducing disputes caused by information asymmetry. On the other hand, the AI-automated review mechanism efficiently handles appeal requests, quickly verifies the accuracy of quality control reports, reduces the pressure and cost of manual review, and only handles disputed items with discrepancies manually, significantly improving appeal processing efficiency and the objectivity of quality control results. Simultaneously, the archiving of reviewed reports and the accumulation of high-quality cases ensure the traceability of quality control data and, in turn, empower service standard optimization and personnel training, forming a continuous iterative closed loop for service quality control.

[0132] In some optional implementations of this embodiment, the method further includes: calibrating the discrepancies between the review results and the quality control report in response to discrepancies; regenerating the quality control report based on the calibrated discrepancies; and archiving the regenerated quality control report.

[0133] After the service personnel file an appeal, the AI ​​system automatically reviews the original quality control report. If there are discrepancies between the review results and the original quality control report (such as inconsistencies in SOP compliance judgments or doubts about the basis for deductions), the system automatically marks the discrepancies and transfers relevant information, including the points of contention, the original data, and the review process, to the calibration stage.

[0134] Quality control personnel receive discrepancies in the process and, in conjunction with the original service recordings, image data, SOP quality control rules, and user feedback, confirm and calibrate the discrepancies: verify whether the AI ​​review logic is reasonable, whether there are any misjudgments or omissions in the original quality control judgment, re-determine the compliance of the disputed points, and correct any erroneous quality control conclusions and deduction criteria.

[0135] Based on the discrepancies identified during calibration by quality control personnel, the original quality control report is updated: the quality control results for the discrepancies are corrected, the service score is recalculated, calibration instructions are supplemented, and a new quality control report is generated. Finally, the regenerated quality control report is archived in the quality control database as the final basis for service quality assessment.

[0136] This solution addresses discrepancies identified by AI-based review by calibrating and correcting quality control judgment biases. Based on the calibration results, a new quality control report is generated and archived, establishing a closed-loop mechanism of "AI initial judgment - appeal review - manual calibration - report update." On one hand, the calibration process effectively mitigates potential misjudgments and omissions in AI-based quality control, improving the accuracy and impartiality of the quality control report and protecting the rights of service personnel. On the other hand, regenerating and archiving the quality control report based on the calibration results ensures that the final conclusions of the service quality assessment are traceable and verifiable, providing an objective and reliable basis for service personnel evaluation, dispute resolution, and platform management. It also provides real feedback data for subsequent optimization of the AI ​​quality control model and SOP rules, driving continuous iteration and optimization of the quality control system.

[0137] In some optional implementations of this embodiment, the method further includes: optimizing the standard operating procedure based on the quality control report in response to the quality control report meeting predetermined conditions.

[0138] Regularly perform batch analysis on archived quality control reports to filter out reports that meet predetermined criteria. For example, reports that frequently occur common non-compliance items, user feedback on common experience issues, AI judgment biases discovered through multiple appeals and reviews, and quality control records corresponding to high-risk events, etc., serve as the triggering basis for SOP optimization.

[0139] For the selected quality control reports, the system combines service process data, AI review results, and manual calibration records to analyze the weaknesses in the standard operating procedures (SOPs): such as a consistently low execution rate for a certain quality control point, a high AI misjudgment rate under the guidance of certain prompts, and poor user experience feedback for a certain step.

[0140] Based on the analysis results, the system can use a large language model to assist in generating SOP optimization suggestions, such as: adjusting the descriptions and prompts of quality control points, adding detailed steps for key operations, adding user communication guidelines or risk notification procedures, and optimizing scoring rules and deduction criteria. Administrators can review and adjust these optimization suggestions, ultimately updating and generating a new SOP version, which is then synchronized to the quality control system and service execution system.

[0141] The optimized SOP will take effect in subsequent services. Service personnel will perform services according to the new version of the SOP, and the quality control system will also conduct AI quality inspections according to the updated quality control points and rules, so as to achieve continuous iteration of service processes and quality control standards.

[0142] This solution triggers the optimization of standard operating procedures (SOPs) through quality control reports, forming a complete closed loop of "service execution → AI quality control → report archiving → process optimization." The quality control reports provide accurate and comprehensive data support for SOP optimization, precisely identifying weak points, high-frequency issues, and user pain points in the process, making optimization more targeted and avoiding the blindness of experience-based adjustments. Continuous optimization based on quality control reports can continuously improve service standards and quality control rules, enhance the executability of SOPs and the accuracy of AI quality control, promote the standardization and refinement of service processes, reduce service disputes and user complaints caused by process defects, and improve overall service quality and platform operational efficiency.

[0143] Figure 4d This example shows a terminal display interface at a nurse's home visit service location: 1. High-risk warning interface: Highlights red warnings, suitable for serious risks such as failure to wash hands or verify information.

[0144] 2. Standard Action Demonstration Interface: Suitable for demonstrating standardized procedures such as disinfection, puncture, and dressing change.

[0145] 3. Multimedia error correction interface: suitable for video, audio, and text / image comparison and error correction.

[0146] See also Figure 4e , Figure 4e This is a schematic diagram illustrating an application scenario of the service quality control method according to this embodiment. Figure 4e In the application scenario, the specific process is as follows: 1. Establish SOPs and quality control points on the operations side. Operations personnel create Standard Operating Procedures (SOPs) in the backend, defining standard actions and corresponding quality control points for the entire service process. Taking "pre-service disinfection" as an example, they configure action descriptions and quality control point prompts (such as "service personnel perform disinfection, handwashing, etc., mentioning words like 'using the restroom, toilet, rinsing hands, washing hands, hand sanitizing, hand disinfectant, and using hand soap or hand sanitizer,' and bringing appropriate items that can be 'disinfected,' as well as phrases like 'Can I use the restroom?' 'I'll disinfect here first,' etc."). They also configure deduction rules, score weights (e.g., the quality control point is worth 10 points), and set speech recognition error tolerance rules, supporting homophone and synonym matching.

[0147] 2. The business system generates a quality control sheet.

[0148] Once the Standard Operating Procedure (SOP) is created, the business system automatically generates a quality control sheet for the corresponding service project, synchronizing the SOP standard actions, quality control point rules, and scoring rules to the quality control system, providing a basis for subsequent service execution and quality inspection.

[0149] 3. The nurse contacts the patient by phone.

[0150] After receiving an order, the nurse will conduct pre-service communication according to the SOP requirements, contact the patient by phone to confirm the service time, location and basic information of the user, and simultaneously inform the patient of the risks before the service.

[0151] 4. The nurse arrives at the user's home.

[0152] The nurse arrived at the user's service location at the agreed time to prepare for the home nursing service.

[0153] 5. The nurse receives and uploads the audio.

[0154] The nurse's app enables real-time recording to collect all audio data of conversations during the service process and uploads it to the quality control system according to preset rules.

[0155] 6. Audio recording editing and merging in the quality control system

[0156] The quality control system receives audio data uploaded by nurses, performs intelligent editing (removing silent segments and invalid noise) and merging of the recordings to generate complete service recording files.

[0157] 7. Upload audio files to OSS storage

[0158] The processed complete audio file is uploaded to the object storage (OSS) system for persistent storage, and used for subsequent speech recognition and quality inspection backtracking.

[0159] 8. ASR service speech recognition, translation, and recording

[0160] The quality control system calls the ASR speech recognition service to transcribe the audio file into structured text data, and at the same time recognizes key information in the dialogue content, such as whether the word "disinfection" is mentioned.

[0161] 9. Perform SOP quality control point verification on the large model.

[0162] The quality control system inputs the ASR transcribed text, SOP quality control point prompts, and general project prompts into the large model. The large model then verifies each line of the text according to preset rules, checking whether the service process has completed all quality control point actions as required by the SOP. Taking "pre-service disinfection" as an example, if the text does not mention the relevant content, it is determined that the action has not been performed.

[0163] 10. The quality control system determines the quality control results and deducts points accordingly.

[0164] Based on the verification results of the large model, the quality control system deducts points for non-compliant quality control points according to the preset deduction rules and score weights. For example, if "disinfection is not mentioned", 10 points will be deducted from the corresponding quality control point.

[0165] 11. End of nurse-side service recording

[0166] Once the nurse completes the service and ends the audio recording, the nurse's app stops recording and completes the on-site data collection for this service.

[0167] 12. Service recording summary and industry knowledge base analysis

[0168] The quality control system, in conjunction with an industry knowledge base, supplements and analyzes complete service recording texts to generate service summaries, including service process compliance, medication recommendations, and nursing guidelines.

[0169] 13. The quality control system pushes summaries to the nurses' terminals.

[0170] The system will push the generated service summary and preliminary quality control results to the nurse's app, allowing nurses to view the execution status and quality control feedback of this service.

[0171] 14. The nurse provides guidance to the patient based on the summary.

[0172] Based on the service summaries pushed by the system, nurses provide follow-up nursing guidance to patients, such as medication methods, precautions, and handling of abnormal situations.

[0173] 15. AI intelligently creates service questionnaires and pushes them to users.

[0174] After the service is completed, the system automatically generates a service quality questionnaire and pushes it to the user's end to collect the user's satisfaction and feedback on the service.

[0175] 16. User submits feedback information

[0176] Users fill out and submit service quality feedback on their terminal devices, including evaluations of service attitude, professionalism, and operational standardization.

[0177] 17. The system integrates multi-dimensional data to generate the final quality control results.

[0178] The quality control system combines the ASR identification results of the service process, the SOP verification results of the large model, and user feedback information to calculate the nurse's total quality control score for this service and generate a complete quality control report.

[0179] 18. The quality control system sends the quality control results to the nurses' terminals.

[0180] The final quality control report is pushed to the nurse's app, where nurses can view detailed quality control information, deductions, and user feedback.

[0181] 19. A nurse filed an appeal against the quality control results.

[0182] If a nurse disagrees with the quality control results, they can file an appeal through the nurse app, submitting the reasons for the appeal and relevant explanations.

[0183] 20. Large models automatically undergo the first round of intelligent review.

[0184] After receiving the appeal request, the quality control system calls the big model to conduct multiple rounds of intelligent review of the disputed items in the quality control report, and re-verifies the ASR text, quality control point rules and service execution.

[0185] 21. Record the verification results and remind quality control personnel to perform manual calibration.

[0186] If the AI ​​review finds discrepancies in the original quality control results, the system automatically marks the points of contention and records the review results, reminding quality control personnel to perform manual calibration.

[0187] 22. Quality control personnel manually calibrate any discrepancies.

[0188] Quality control personnel combine original recordings, service records, SOP rules, and AI review results to manually confirm disputed items, correct erroneous quality control judgments, and adjust the basis for deductions.

[0189] 23. The quality control system returns the appeal results to the nurses' terminals.

[0190] After manual calibration is completed, the system will push the final verification result to the nurse's end, informing the nurse of the appeal processing result and the updated quality control report.

[0191] 24. The appeal process is complete; management can view and provide information.

[0192] The appeal process is closed-loop, the quality control report is archived based on the final calibration results, and managers can view the quality control data on the management terminal to conduct targeted service standard education and training for nurses.

[0193] 25. Large-scale model recognition and accumulation of high-quality cases.

[0194] The system analyzes and identifies service cases with high quality control scores, good user feedback, and standardized process execution through a large model, and marks them as high-quality service cases.

[0195] 26. High-quality case studies are incorporated into business systems and knowledge bases.

[0196] High-quality case studies are stored in the business system and knowledge base, including complete data such as service recordings, quality control reports, and user feedback, as materials for subsequent service standard optimization and nurse training.

[0197] 27. Automatically generate new project SOPs based on high-quality case studies.

[0198] The system combines accumulated high-quality case studies and uses large models to help generate SOP processes and quality control points for new service projects, enabling automatic iteration and continuous optimization of service standards and completing the closed loop of the quality control system.

[0199] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a quality control device based on service site data. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0200] like Figure 5 As shown, the quality control device 500 based on service site data in this embodiment includes: a receiving unit 501, configured to receive multimedia data uploaded in real time by a terminal device during the service process at the service site; the multimedia data includes one or more of audio data, image data, and video data; a determining unit 502, configured to determine the content to be prompted during the service process based on the multimedia data; a generating unit 503, configured to generate prompt information according to the content to be prompted; and a prompting unit 504, configured to send the prompt information to the terminal device, so that the terminal device can issue prompts to service personnel through one or more of images, text, sound, and video.

[0201] In this embodiment, the specific processing of the receiving unit 501, determining unit 502, generating unit 503, and prompting unit 504 of the quality control device 500 based on service site data can be referred to Figure 2 The corresponding steps are 201, 202, 203 and 204 in the embodiment.

[0202] In some optional implementations of this embodiment, the determining unit 502 is further configured to: identify multimedia data to obtain identification results; acquire a pre-created standard operating procedure for the service site, wherein the standard operating procedure includes: standard actions and quality control points for the standard actions; input the identification results and preset action prompts into a large language model to determine the content to be prompted, wherein the action prompts are used to guide the large language model to determine the next standard action and precautions as the content to be prompted by analyzing the identification results, so that the action performed by the service provider in accordance with the content to be prompted meets the requirements of the quality control points.

[0203] In some optional implementations of this embodiment, the generation unit 503 is further configured to: generate multimedia demonstration data of standard actions based on the content to be prompted, wherein the multimedia demonstration data includes one or more of audio data, image data, and video data.

[0204] In some optional implementations of this embodiment, the determining unit 502 is further configured to: identify multimedia data to obtain identification results; input the identification results and preset operation specification prompts into a large language model to determine whether there are risks in the service process, wherein the operation specification prompts are used to guide the large language model to analyze whether the service provider's operation conforms to the standard operation specification; and in response to non-compliance with the standard operation specification, determine the content to be prompted based on the non-compliance operation and the corresponding standard operation.

[0205] In some optional implementations of this embodiment, the generation unit 503 is further configured to: generate multimedia error correction data to correct operations that do not conform to standard operating procedures based on the content to be prompted, so that the service provider can correct operations that do not conform to standard operating procedures, wherein the multimedia error correction data includes one or more of audio data, image data and video data.

[0206] In some optional implementations of this embodiment, the device 500 further includes a quality inspection unit (not shown in the figures), configured to: in response to the end of the service, identify the complete multimedia data of the service process and obtain an identification result; perform a compliance check on the standard operating procedure based on the identification result and obtain an inspection result, wherein the standard operating procedure includes: standard actions and quality control points of the standard actions; send a service quality questionnaire to the terminal of the service recipient; and in response to receiving feedback information from the service recipient regarding the service quality questionnaire, generate a quality control report based on the feedback information and the inspection result.

[0207] In some optional implementations of this embodiment, the quality inspection unit is further configured to: send a quality control report to the terminal device; archive the quality control report in response to receiving a quality control report confirmation message from the terminal device; review the quality control report based on an artificial intelligence model in response to receiving an appeal request from the terminal device; and archive the quality control report in response to the review result being consistent with the quality control report.

[0208] In some optional implementations of this embodiment, the quality inspection unit is further configured to: calibrate the discrepancies between the review results and the quality control report in response to discrepancies; regenerate the quality control report based on the calibrated discrepancies and archive the regenerated quality control report; and optimize the standard operating procedure based on the quality control report in response to the quality control report meeting predetermined conditions.

[0209] In some optional implementations of this embodiment, the prompting unit 504 is further configured to: in response to the end of the service, identify the complete multimedia data of the service process and obtain the identification result; generate guidance information based on the identification result and the goal of the service item; and send the guidance information to the terminal device and / or the terminal of the service object.

[0210] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0211] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0212] An electronic device includes: one or more processors; and a storage device having one or more computer programs stored thereon, which, when executed by the one or more processors, cause the one or more processors to implement the method described in process 200 or 400.

[0213] A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in process 200 or 400.

[0214] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0215] like Figure 6As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0216] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0217] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0218] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0219] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0220] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or electronic devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0221] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0222] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0223] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be servers in distributed systems or servers incorporating blockchain technology. Servers can also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0224] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0225] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A quality control method based on service site data, characterized in that, include: The system receives multimedia data uploaded in real time by terminal devices during the service process at the service site, wherein the multimedia data includes one or more of audio data, image data, and video data. Based on the multimedia data, determine the content to be prompted during the service process; Based on the content to be prompted, generate prompt information; The prompt information is sent to the terminal device, so that the terminal device can issue a prompt to the service personnel through one or more of the following: images, text, sound, and video.

2. The method according to claim 1, wherein, The step of determining the content to be prompted during the service process based on the multimedia data includes: The multimedia data is identified to obtain the identification result; Obtain the pre-created standard operating procedure for the service site, wherein the standard operating procedure includes: standard actions and quality control points for the standard actions; The recognition results and preset action prompts are input into a large language model to determine the content to be prompted. The action prompts are used to guide the large language model to analyze the recognition results and determine the standard action and precautions for the next step as the content to be prompted, so that the service provider can perform the action according to the content to be prompted to meet the requirements of the quality control point.

3. The method according to claim 2, wherein, The step of generating prompt information based on the content to be prompted includes: Based on the content to be prompted, multimedia demonstration data of the standard action is generated, and the multimedia demonstration data includes one or more of audio data, image data, and video data.

4. The method according to claim 1, wherein, The step of determining the content to be prompted during the service process based on the multimedia data includes: identifying the multimedia data to obtain an identification result; The identification results and preset operation specification prompts are input into the large language model to determine whether there are risks in the service process. The operation specification prompts are used to guide the large language model to analyze whether the service provider's operation conforms to the standard operation specifications. In response to non-compliance with standard operating procedures (SOPs), determine the prompt content based on the non-compliant operation and the corresponding standard operating procedure.

5. The method according to claim 4, wherein, The step of generating prompt information based on the content to be prompted includes: Based on the content to be prompted, multimedia error correction data is generated to correct operations that do not conform to standard operating procedures, so that the service provider can correct operations that do not conform to standard operating procedures. The multimedia error correction data includes one or more of audio data, image data, and video data.

6. The method according to claim 1, further comprising: In response to the end of the service, the complete multimedia data of the service process is identified to obtain the identification result; Based on the identification results, a compliance check of the standard operating procedure is performed to obtain the check results. The standard operating procedure includes: standard actions and quality control points for the standard actions. Send service quality questionnaires to the terminals of service recipients; In response to receiving feedback from the service recipient regarding the service quality questionnaire, a quality control report is generated based on the feedback and the inspection results.

7. The method according to claim 6, further comprising: Send the quality control report to the terminal device; In response to receiving a quality control report confirmation message from the terminal device, the quality control report is archived; In response to receiving an appeal request from the terminal device, the quality control report is reviewed based on an artificial intelligence model; If the verification result is consistent with the quality control report, the quality control report is archived.

8. The method according to claim 7, further comprising: In response to discrepancies between the verification results and the quality control report, the discrepancies are calibrated. Regenerate the quality control report based on the differences after calibration, and archive the regenerated quality control report; In response to the quality control report meeting predetermined conditions, the standard operating procedure is optimized based on the quality control report.

9. The method according to any one of claims 1-8, further comprising: In response to the end of the service, the complete multimedia data of the service process is identified to obtain the identification result; Based on the identification results and the goals of the service projects, guidance information is generated; The guidance information is sent to the terminal device and / or the terminal of the service recipient.

10. A quality control device based on service site data, characterized in that, include: The receiving unit is configured to receive multimedia data uploaded in real time by the terminal device during the service process at the service site; The multimedia data includes one or more of the following: audio data, image data, and video data; The determining unit is configured to determine the content to be prompted during the service process based on the multimedia data; The generation unit is configured to generate prompt information based on the content to be prompted; The prompting unit is configured to send the prompting information to the terminal device, so that the terminal device can issue a prompt to the service personnel through one or more of the following: images, text, sound, and video.

11. An electronic device, comprising: One or more processors; Storage device, on which one or more computer programs are stored, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

12. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images