Intelligent management method and system for home care
The intelligent home care management system uses audio and video data to objectively evaluate service quality by identifying biased reviews, enhancing the authenticity and accuracy of home care service assessments.
Patent Information
- Application Number
- CN202411405703.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-10-10
AI Technical Summary
Existing methods for evaluating the quality of home care services rely heavily on user satisfaction scores, which can be influenced by biased or malicious reviews, leading to inaccurate assessments of service quality.
An intelligent management system that utilizes audio and video data from home care tasks to extract and fuse features, perform clustering, and match evaluation text features with operation video clips to determine a nursing evaluation weight factor, adjusting evaluation data to identify forced positive or malicious negative reviews.
The system provides more authentic quality evaluations by distinguishing between genuine and biased reviews, improving the objectivity and accuracy of home care service assessments.
Smart Images

Figure CN118916660B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to an intelligent management method and system for home care. Background Art
[0002] In order to promote the high-level development of "Internet + Nursing Services", medical institutions in various provinces have established a training system for home nurses for different specialties. The aim is to improve the level of home care services provided by medical staff through a series of professional training courses. After systematic training and strict access assessment, qualified medical staff will obtain home care service qualifications and be allowed to provide home care services to users.
[0003] In order to effectively improve the quality of home care services, it is crucial to pay attention to user satisfaction. In related technologies, the evaluation process of the quality of home care services provided by medical staff usually includes: after the medical staff completes the home care service, the user evaluates the service, and then the quality of the medical staff's service is determined based on these evaluation results. However, the method of determining the quality of medical services based on user evaluation data in related technologies needs to be improved, especially for situations that deviate from objective facts, such as forcing positive reviews or malicious negative reviews. Summary of the invention
[0004] The present application provides an intelligent management method and system for home care, which solves the technical problem of insufficient authenticity of medical care service quality evaluation in related technologies, and achieves the technical effect of obtaining a more authentic medical care service quality evaluation.
[0005] In order to achieve the above objectives, the main technical solutions adopted in this application include:
[0006] In a first aspect, an embodiment of the present application provides a method for intelligent management of home care, which is applied to a platform management end of a home care intelligent management system, wherein the home care intelligent management system further comprises a user end and a medical end which are communicatively connected to the platform management end; the user end uploads image data of a location where a service item occurs to the platform management end before a home care task is started; the medical end uploads audio and video data of the care task to the platform management end after the home care task is completed; the home care task comprises at least one service item; the method comprises:
[0007] Perform feature extraction and feature fusion based on the audio and video data of the nursing task to obtain audio and video fusion features;
[0008] Performing clustering processing according to the audio and video fusion features to obtain multiple fusion feature sets; and intercepting a nursing operation video clip of at least one service item from the nursing task audio and video data according to the image data of the location where the service item occurs and the multiple fusion feature sets;
[0009] Feature matching is performed based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip;
[0010] Determining the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor; wherein the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review;
[0011] The home care evaluation data is adjusted according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain true quality evaluation data of the home care task.
[0012] In a second aspect, an embodiment of the present application provides a home care intelligent management device, which is applied to a platform management end of a home care intelligent management system, wherein the home care intelligent management system further comprises a user end and a medical end which are communicatively connected to the platform management end; the user end uploads image data of a location where a service item occurs to the platform management end before a home care task is started; the medical end uploads audio and video data of the care task to the platform management end after the home care task is completed; the home care task comprises at least one service item; the device comprises:
[0013] A feature extraction and fusion module is used to extract and fuse features based on the audio and video data of the nursing task to obtain audio and video fusion features;
[0014] an operation video acquisition module, configured to perform clustering processing according to the audio and video fusion features to obtain a plurality of fusion feature sets; and to intercept a nursing operation video clip of at least one service item from the nursing task audio and video data according to the image data of the location where the service item occurs and the plurality of fusion feature sets;
[0015] A text-video matching module, used for performing feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip, to obtain a nursing evaluation weight factor; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip;
[0016] An evaluation category determination module, used to determine the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor; wherein the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review;
[0017] An evaluation data adjustment module is used to adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain true quality evaluation data of the home care task.
[0018] In a third aspect, an embodiment of the present application provides a computer device, including:
[0019] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in any of the above embodiments by executing the computer instructions.
[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.
[0021] In a fifth aspect, an embodiment of the present application provides a home care intelligent management system, the home care intelligent management system includes a platform management end, a user end and a medical end that are communicatively connected to the platform management end; wherein:
[0022] The user terminal is used to upload the image data of the location of the service project to the platform management terminal before the home care task is started, and to upload the home care evaluation data to the platform management terminal after the home care task is completed; the home care task includes at least one service project;
[0023] The medical end is used to upload the audio and video data of the nursing task to the platform management end after the home nursing task is completed;
[0024] The platform management end is used to perform feature extraction and feature fusion based on the nursing task audio and video data to obtain audio and video fusion features, perform clustering processing based on the audio and video fusion features to obtain multiple fusion feature sets; and based on the image data of the location where the service project occurs and the multiple fusion feature sets, intercept a nursing operation video clip of at least one service project from the nursing task audio and video data; perform feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; and adjust the home care evaluation data according to the nursing evaluation weight factor to obtain true quality evaluation data of the home care task.
[0025] In an embodiment of the present application, audio and video fusion features are obtained based on feature extraction and fusion processing of audio and video data of nursing tasks, and nursing operation video clips corresponding to each service item are obtained based on clustering processing of these fusion features. Furthermore, a nursing evaluation weight factor is obtained based on the text features in the nursing evaluation text data and the video features of the nursing operation video clip. The nursing evaluation weight factor is used to reflect the consistency between the home care evaluation data and the nursing operation video clip, and the home care evaluation data is adjusted using the nursing evaluation weight factor to obtain more real and objective home care service evaluation data. Compared with the method based solely on user evaluation in the related art, it effectively improves the objectivity of the evaluation results, helps to obtain a more realistic quality of medical and nursing services, and thus improves the level of home care services. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0027] Figure 1a A schematic diagram of an intelligent home care management system provided in an embodiment of this specification;
[0028] Figure 1b A schematic diagram of an operation interface for a user terminal to reserve a home care service according to an embodiment of this specification;
[0029] Figure 1c A flowchart of the intelligent home care management method provided in the embodiments of this specification;
[0030] Figure 1d A schematic diagram of a user-side evaluation interface provided in an embodiment of this specification;
[0031] Figure 2 A flowchart of a method for intelligent management of home care provided in one embodiment of this specification;
[0032] Figure 3 A flowchart of a home care intelligent management method provided by another embodiment of this specification;
[0033] Figure 4 A schematic diagram of an intelligent home care management device provided in an embodiment of this specification;
[0034] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of this specification. DETAILED DESCRIPTION
[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0036] In the relevant technology, the quality of home care services is mainly evaluated by user satisfaction, and user satisfaction is fed back by scoring service items. However, in real life, due to various reasons, the user's rating may not match the actual nursing service situation. For example, although the user is not satisfied with the service, he may give a false positive review under the lobbying of medical staff. This situation is not conducive to obtaining real user satisfaction and improving the quality of home care services. For another example, some users give malicious bad reviews to medical staff, which cannot truly reflect the quality of home care provided by medical staff.
[0037] Therefore, this application scenario example proposes a home care intelligent management method, please refer to Figure 1a , this method is applied to the platform management end 120 of the home care intelligent management system, and the system also includes a user end 110 and a medical end 130 that are connected to the platform management end 120 in communication. Among them, the platform management end 120 is responsible for the overall management of the system, which may include functional modules such as registration and review of users and medical staff, service information management, and service quality management to ensure the normal operation of the system and the security of data. The user end 110 may be a client provided to users, which has functional modules such as submitting nursing needs, viewing service progress, evaluating nursing services, and managing personal information. The medical end 130 may be a client provided to medical staff, which has functional modules such as receiving nursing requests, viewing service information, updating service status, and managing personal information.
[0038] For a schematic diagram of the nursing requirement selection interface of the user terminal 110, please refer to Figure 1b , users can make an appointment for a single home care service or for multiple home care services at the same time. After the payment is successful, the home care service request is submitted. It is understandable that when submitting the care needs, the user should also select the date and time period of the appointment. You can specify a medical staff or not. The system will assign the order, and medical staff with home care service qualifications can accept the order. When providing home care services, medical staff will record the entire process of audio and video. At the end of the service, the recorded audio and video files will be uploaded to the platform management terminal 120 through the medical terminal 130. At the end of the service, the user can evaluate the service through the user terminal 110, and the evaluation data can be uploaded to the platform management terminal 120.
[0039] In this scenario example, before the home care task, it is necessary not only to disturb the user who has booked the home care service as little as possible, but also to ensure that the caregiver can accurately arrive at the execution site of the service project. If it is simply based on the geographic location information and house number information, the accuracy of the geographic location information is not enough, and the house number information has a certain degree of duplication. Therefore, the home care demand data uploaded by the user terminal 110 includes the designated medical staff identification, the first geographic location information of the service project, and the first porch image data. The first porch image data can accurately reflect the actual situation of the execution site of the service project. The designated medical staff identification can be used to refer to the medical staff designated by the user who is qualified to provide the scheduled service project.
[0040] When the medical staff believes that they have arrived at the execution site of the service project, the second geographic information location of the medical staff is compared with the first geographic location information of the service project reserved by the user. If the two are consistent, the permission to take pictures of the porch environment of the execution site can be enabled. The pictures can be two images, a porch image and a face image of the medical staff, or one image including the porch environment of the execution site and the face of the medical staff.
[0041] Since the medical staff can take a photo of the porch of the execution site where they are located and upload the second porch image to the platform management terminal 120, it should be noted that the second porch image can include the face of the medical staff. The platform management terminal 120 can obtain the second porch image data and the medical staff's facial features through image analysis and feature extraction.
[0042] The platform management terminal 120 pre-stores the identification feature correspondence between the medical staff identification and the medical staff facial features. After the platform management terminal 120 determines the medical staff facial features, it searches the identification feature correspondence relationship according to the designated medical staff identification to obtain the target facial features; the medical staff facial features uploaded by the medical staff terminal 130 are compared with the target facial features corresponding to the designated medical staff identification to determine whether the similarity between the target facial features and the medical staff facial features exceeds the similarity threshold. The platform management terminal 120 compares the first porch image data with the second porch image data to conduct a service address review. If it is detected that the first porch image data and the second porch image data are taken for the same porch, the review is passed and an arrival reminder message is sent to the user terminal 110. The medical staff no longer needs to knock on the door or ring the doorbell, and the user can also know that the medical staff has arrived at the execution site of the service project.
[0043] In this scenario example, in order to protect user privacy, the platform management end 120 pre-stores the corresponding relationship of identification features, and the corresponding relationship of identification features can be generated by interpolation operation based on the medical staff identification and the medical staff facial features. The user end 110 uses a private key to homomorphically encrypt the designated medical staff identification to obtain the ciphertext personnel identification. The user end 110 sends the ciphertext personnel identification to the platform management end 120. The platform management end 120 uses a polynomial function to perform homomorphic calculation on the ciphertext personnel identification to obtain the ciphertext facial features corresponding to the designated medical staff identification. The medical end 130 uses a private key to homomorphically encrypt the medical staff facial features of the medical staff who arrive at the service project execution site to obtain the ciphertext features to be verified. The similarity between the ciphertext features to be verified and the ciphertext facial features corresponding to the designated medical staff identification is calculated. If the similarity exceeds the similarity threshold, the service address is reviewed according to the second porch image data and the first porch image data. The privacy data such as medical staff data and facial data are invisible to the ciphertext of the platform management end 120, thereby improving data security. For example, the homomorphic encryption scheme may adopt the Brakerski / Fan-Vercauter encryption system (BFV), which utilizes the fully homomorphic property of the BFV algorithm and performs polynomial iterations in the ciphertext state.
[0044] In this scenario example, in order to objectively and truly determine the quality of home care services, on the one hand, the execution process of the home care task is recorded by the medical terminal 130, so that after the home care task is completed, the audio and video data of the nursing task uploaded by the medical terminal 130 is obtained; on the other hand, after the home care task is completed, the home care evaluation data uploaded by the user terminal 110 is obtained. Since the entire execution process of the home care task includes a variety of video clips, such as the process of the user greeting the nursing staff, the process of the user chatting with the nursing staff, and the execution process of the service items booked by the user. However, only the execution process video clips of the service items can reflect the quality of home care services, so it is necessary to intercept the nursing operation video clips of the service items from such a large variety of redundant complete execution process videos. Furthermore, although the complete execution process video includes various video clips, different video clips may occur at different locations, such as the greeting process can occur at the entrance, the chatting process can occur at the sofa in the living room, and the touch service project can occur on the baby care table. Therefore, different video clips have different occurrence scenes, and the video frames in the video clips belonging to the same service project have correlation and similarity. Then, the audio and video fusion features corresponding to the audio and video data of the nursing task are clustered to obtain multiple fusion feature sets. In addition, because different video clips have different occurrence scenes, and the user end uploads the image data of the service project's occurrence location to the platform management end before the home care task is started, the nursing operation video clips of each service project can be accurately obtained through clustering processing and the image data of the service project's occurrence location. By verifying and adjusting the authenticity of the home care evaluation data through the similarity of images and texts between the home care evaluation data and the nursing operation video clips, the real quality evaluation data of the home care tasks are obtained. By using the nursing operation video clips to evaluate and adjust the home care evaluation data, it is possible to timely identify the medical staff's forced positive reviews and the users' malicious negative reviews, and reasonably protect the respective rights and interests of the medical staff and users.
[0045] According to an embodiment of the present application, an embodiment of a method for intelligent management of home care is provided. The method is applied to a platform management end of a home care intelligent management system. The home care intelligent management system also includes a user end and a medical end that are connected to the platform management end in communication. The user end uploads image data of the location where the service item occurs to the platform management end before the home care task is started. The medical end uploads audio and video data of the care task to the platform management end after the home care task is completed. The home care task includes at least one service item. For details, please refer to Figure 1c ,The method may include the following steps:
[0046] S101. Perform feature extraction and feature fusion based on the audio and video data of the nursing task to obtain audio and video fusion features.
[0047] Among them, the medical terminal 130 performs video shooting during the execution of the home care task to obtain the audio and video data of the nursing task. Specifically, the audio and video data of the nursing task includes data in two modes, audio and video. Feature extraction can be performed on the audio mode to obtain nursing audio features, and feature extraction can be performed on the video mode to obtain nursing video features. The nursing audio features and nursing video features are fused to obtain audio and video fusion features. It should be noted that the audio data can also be converted into text data and then feature extraction is performed, and the obtained text features are fused with the video features to obtain audio and video fusion features.
[0048] S103, performing clustering processing based on the audio and video fusion features to obtain multiple fusion feature sets; and intercepting a nursing operation video clip of at least one service item from the nursing task audio and video data based on the image data of the service item's occurrence location and the multiple fusion feature sets.
[0049] In some cases, for different locations or different themes of events, the audio and video content belonging to the same event have similarities and correlations. Specifically in this embodiment, the video frames corresponding to different service items have the same location. Therefore, first, the audio and video fusion features can be clustered, and the audio and video fusion features with similarities and correlations can be classified into the same cluster to obtain multiple fusion feature sets; secondly, the audio and video data corresponding to the service items are determined in multiple fusion feature sets using the image data of the location where the service items occur, so as to intercept the nursing operation video clips of at least one service item from the nursing task audio and video data.
[0050] S105 , performing feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clips to obtain a nursing evaluation weight factor.
[0051] Among them, the nursing evaluation weight factor is used to characterize the consistency between the home nursing evaluation data and the nursing operation video clip, or in other words, the nursing evaluation weight factor is used to characterize the consistency between the home nursing evaluation data and the nursing operation video clip. The nursing operation video clip objectively records the true situation of the entire process, while the home nursing evaluation data may be consistent with the true situation due to different situations, or may deviate from the true situation to varying degrees. Therefore, in this embodiment, it is proposed to use the nursing evaluation weight factor to describe the difference or consistency between the nursing operation video clip and the home nursing evaluation data.
[0052] Specifically, text extraction is performed on the home care evaluation data of the home care task to obtain the nursing evaluation text data in the home care evaluation data. Text feature extraction is performed on the nursing evaluation text data to obtain evaluation text features. Feature extraction is performed on the video frames in the nursing operation video clip to obtain operation video features, and feature matching is performed based on the evaluation text features and the operation video features to obtain the nursing evaluation weight factor.
[0053] S107. Determine the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor.
[0054] The evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review. The total evaluation score can be understood as the full score of the home care task, for example, the full score can be 20 points. The home care evaluation data can be 18 points, and the difference between the two can be 2 points.
[0055] In some cases, at the insistence of medical staff, users give full marks to home care tasks out of kindness. In other cases, the home care services provided by medical staff are good but receive malicious negative reviews from users. Therefore, the above situations are identified by combining the difference between the total evaluation score of the home care task and the home care evaluation data, as well as the nursing evaluation weight factor.
[0056] Specifically, for example, the home care evaluation data shows that the score of medical staff A is good, but the similarity between the home care evaluation data and the nursing operation video clip of medical staff A is very low, and the nursing operation video clip is an objective record of the facts of the nursing process. Therefore, the nursing quality of medical staff A may not be high, and the home care evaluation data may also be forced to be good. In order to further verify, the difference between the total evaluation score of the home care task and the home care evaluation data is determined. If the difference is less than the preset difference threshold, it means that the home care evaluation data is close to full marks, which further proves that the home care evaluation data is forced to be good, thereby determining the evaluation category of the home care evaluation data.
[0057] If the nursing evaluation weight factor is relatively small, such as less than 0.5, it indicates that the home care evaluation data deviates from the nursing operation video clip. For example, the home care evaluation data shows that the score of medical staff A is a bad review, but the similarity between the home care evaluation data and the nursing operation video clip of medical staff A is very low, and the nursing operation video clip is an objective record of the facts of the nursing process. Therefore, the nursing quality of medical staff A should be good, and the home care evaluation data may be a malicious bad review. In order to further verify, the difference between the total evaluation score of the home care task and the home care evaluation data is determined. If the difference exceeds the preset difference threshold, it indicates that the home care evaluation data is close to the minimum score, which further proves that the home care evaluation data is a malicious bad review, thereby determining the evaluation category of the home care evaluation data.
[0058] If the nursing evaluation weight factor is relatively large, for example, exceeding 0.8, it indicates that the home nursing evaluation data is consistent with the nursing operation video clip. Since the nursing operation video clip is an objective record of the facts of the nursing process, and the home nursing evaluation data is consistent with the nursing operation video clip, it indicates that the home nursing evaluation data is credible, and it is determined that the evaluation category of the home nursing evaluation data belongs to the real evaluation category.
[0059] S109. Adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain the true quality evaluation data of the home care task.
[0060] Specifically, if the nursing evaluation weight factor indicates that the home care evaluation data is a real evaluation, the nursing evaluation weight factor can be used to fine-tune the home care evaluation data. Alternatively, because the nursing evaluation weight factor indicates that the home care evaluation data is credible, the home care evaluation data can be regarded as the real quality evaluation data of the home care task.
[0061] If the nursing evaluation weight factor indicates that the home care evaluation data deviates from the nursing operation video clips, first determine the evaluation category, then call the adjustment strategy corresponding to the evaluation category, and adjust the home care evaluation data according to the adjustment strategy corresponding to the evaluation category using the total evaluation score and the nursing evaluation weight factor to obtain true quality evaluation data.
[0062] For example, if the full score of the home care task is 20 points, the nursing evaluation weight factor is 0.4, and the home care evaluation data is 18 points, which is a required positive score, then the product of 18 points and the nursing evaluation weight factor is determined to be 7.2, and the difference between the full score of 20 points and the home care evaluation data of 18 points is determined to be 2 points. The product 7.2 and the difference of 2 points are summed to obtain the true quality evaluation data of 9.2 points.
[0063] For example, if the full score of the home care task is 20 points, the nursing evaluation weight factor is 0.2, and the home care evaluation data is 6 points, which is a malicious bad review, then the product of 6 points and the nursing evaluation weight factor is determined to be 1.2, and the difference between the full score of 20 points and the home care evaluation data of 6 points is determined to be 14 points. The product 1.2 and the difference 14 points are summed to obtain the true quality evaluation data of 15.2 points.
[0064] In the above implementation, firstly, the nursing operation video clip of each service item is accurately obtained from the audio and video data of the nursing task through the clustering processing method and the image data of the location where the service item occurs. Secondly, feature matching is performed based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain the nursing evaluation weight factor, and the deviation between the home care evaluation data and the nursing operation video clip is judged by the nursing evaluation weight factor. Then, the evaluation category of the home care evaluation data is identified based on the difference between the nursing evaluation weight factor and the total evaluation score of the home care task and the home care evaluation data, so as to clarify the adjustment strategy for the home care evaluation data, and finally obtain the real quality evaluation data of the home care task, accurately identify forced good reviews and malicious bad reviews, and adjust the home care evaluation data in time, so as to improve the objectivity of the quality evaluation data and truly reflect the user's evaluation of the home care service.
[0065] In some embodiments, the evaluation category of the home care evaluation data is determined based on the difference between the total evaluation score of the home care task and the home care evaluation data and the nursing evaluation weight factor, including: if the nursing evaluation weight factor is less than the preset adjustment threshold, and the difference between the total evaluation score and the home care evaluation data is further determined to exceed the first preset difference threshold, the home care evaluation data is determined to belong to the malicious bad review category. If the nursing evaluation weight factor is less than the preset adjustment threshold, the difference between the total evaluation score and the home care evaluation data is further determined to be less than the second preset difference threshold, and the home care evaluation data is determined to belong to the forced good review category; wherein the second preset difference threshold is greater than the first preset difference threshold.
[0066] According to an embodiment of the present application, an embodiment of an intelligent home care management method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0067] The method and steps for the platform management terminal 120 to intelligently manage the service quality include:
[0068] S110, obtaining home care evaluation data uploaded by the user after completing the home care task, and performing text extraction from the home care evaluation data to obtain care evaluation text data.
[0069] The home care evaluation data is generated by the user terminal 110 in response to the nursing evaluation operation for the home care task on the evaluation questionnaire interactive interface after the home care service is completed. After the platform management terminal 120 obtains the home care evaluation data, it performs text extraction, and the extracted text may be the name text of the home care service and the text of the corresponding evaluation item. It should be noted that in each service evaluation data, the nursing service may be one or more items.
[0070] For example, see Figure 1d , the user terminal 110 evaluates the two services, baby touch 140 and postpartum rehabilitation exercise 150, respectively. After each evaluation item, select the score to be evaluated, fill in other more helpful content for the evaluation, and then submit. In other examples, for each evaluation item, the number of lit five-pointed stars can represent the score to be evaluated. After the platform management terminal 120 obtains the home care evaluation data, it performs text extraction, which can be the text of baby touch 140 and its corresponding evaluation, and the text of postpartum rehabilitation exercise 150 and its corresponding evaluation. It should be noted that the evaluation text can be converted in combination with the text of the evaluation item and the corresponding user score to obtain the final user evaluation text. For example, when the text content of the evaluation item is "the service attitude of the nursing staff", the corresponding user scores 1, 2, 3, 4, and 5 can be converted into "the service attitude of the nursing staff is very poor", "the service attitude of the nursing staff is relatively poor", "the service attitude of the nursing staff is average", "the service attitude of the nursing staff is relatively good", and "the service attitude of the nursing staff is very good".
[0071] S120, obtaining the audio and video data of the nursing task uploaded by the medical staff after the home care task is completed, performing feature extraction and feature fusion based on the audio and video data of the nursing task, and obtaining audio and video fusion features.
[0072] The audio and video data of the nursing task are obtained by collecting video and audio of the execution process of the home nursing task by the medical terminal 130. The fusion feature is used to represent the overall features of the video and the corresponding audio. The fusion feature can be represented by a vector or other forms. For example, the fusion feature can be directly obtained by splicing the video feature and the audio feature, or it can be obtained by extracting the video feature and the audio feature based on the cross-attention mechanism.
[0073] In some cases, more comprehensive information can be provided by two types of data, video and its corresponding audio. Therefore, feature extraction and feature fusion are performed based on the audio and video data of the nursing task to obtain audio and video fusion features. Specifically, the video is encoded to obtain video features, and the audio corresponding to the video is encoded to obtain audio features. The video features and audio features can be encoded to obtain audio and video fusion features.
[0074] S130. Perform clustering processing according to the audio and video fusion features to extract a nursing operation video clip of at least one service item from the nursing task audio and video data.
[0075] Specifically, multiple audio and video segments are involved in the audio and video collection process of home care services, such as infant intellectual care training segments and pregnant women rehabilitation exercise segments. Relatively speaking, the similarity between audio and video in the same video segment is usually higher than the similarity between audio and video in different video segments. Therefore, the fusion features of multiple audio and video can be divided by clustering, and multiple fusion features with high similarity can be divided into the same audio and video segment cluster, and the similarity between different audio and video segment clusters is low, so as to find the hierarchical structure in the audio and video file.
[0076] Exemplarily, the K-Means algorithm can be used to cluster the fusion features of the audio and video of the home care service. The algorithm first selects K random centroids, then assigns each fusion feature to the cluster where the nearest centroid is located, then recalculates the centroid of each cluster, and assigns each fusion feature to the cluster where the nearest centroid is located again. Repeat this process until the centroid no longer changes or reaches the preset maximum number of iterations, and obtain multiple video clips, and determine the nursing operation video clips of each service item in the home care service in the multiple video clips. For example, according to the correlation between the video features corresponding to any video clip and the text description features of the service item, it is determined whether any video clip is a nursing operation video clip of the service item. The text description features of the service item can be the name of the service item, or it can be a description text of the key step operation of the service item. It should be noted that in this embodiment, the nursing operation video clip is determined by means of the technical means of video understanding, which provides an accurate data basis for the subsequent verification and adjustment of the authenticity of the home care evaluation data in combination with the similarity between the pictures and texts of the nursing operation video clips.
[0077] S140 , performing feature matching based on evaluation text features of the nursing evaluation text data and operation video features of the nursing operation video clips to obtain a nursing evaluation weight factor.
[0078] Specifically, feature extraction is performed on the evaluation text of any home care service item to obtain evaluation text features. Feature extraction is performed on the operation video of the nursing operation video clip to obtain operation video features. Feature matching is performed by comparing the similarity between two features or calculating the Euclidean distance, and the feature matching result is used as a nursing evaluation weight factor. Exemplarily, the numerical range of the weight factor can be a value in the range of 0 to 1. In other embodiments, there are multiple service items in a home care service, and feature extraction can be performed on the evaluation text of each service item, and then feature matching is performed between the evaluation text features and the operation video features of each service item to obtain the nursing evaluation weight factor of each service item.
[0079] S150. Adjust the home care evaluation data according to the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task.
[0080] Specifically, since the home care evaluation data may be a favorable comment forced by medical staff or a malicious negative comment by the user, in this embodiment, the home care evaluation data is readjusted using the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task, so that the favorable comments forced by medical staff and the malicious negative comments by the user can be timely identified, and the rights and interests of medical staff and users can be reasonably protected. For example, a home care service includes a service item, which has 4 evaluation items, and the user scores are all 5 points, and the total score is 20 points. The 20 points can be regarded as the home care evaluation data. If it obtains a nursing evaluation weight factor of 0.8, then the real quality evaluation data of the home care service obtained after adjustment can be the product of 20 and 0.8, that is, 16 points. It can be seen that the graphic similarity between the home care evaluation data and the nursing operation video clip is relatively high, and the evaluation data corresponding to the home care service is relatively objective. If it obtains a nursing evaluation weight factor of 0.3, then the real quality evaluation data of the home care service obtained after adjustment can be the product of 20 and 0.3, that is, 6 points. It can be seen that the evaluation data corresponding to this home care service is quite deviated from the facts.
[0081] A home care service includes a service item, which has 4 evaluation items with a total score of 20 points. The user scores are all 2 points, and the total user score is 8 points. The 8 points can be regarded as home care evaluation data. If it obtains a nursing evaluation weight factor of 0.2, it can be seen that the graphic similarity between the home care evaluation data and the nursing operation video clips is relatively low, and the nursing operation video clips cannot be falsified, so it will be considered that the home care evaluation data is generated by the user's malicious bad review operation. In other words, the evaluation data corresponding to the home care service is relatively deviated from the facts. For malicious bad reviews and other situations, the real service quality score can be re-determined based on the total score, nursing evaluation weight factor, and user total score. For example, the real quality evaluation data of the home care service after adjustment can be 18 points, calculated as follows: .
[0082] In the above implementation, in order to objectively and truly determine the quality of home care services, on the one hand, the medical end records the execution process of the home care task through audio and video, so as to obtain the audio and video data of the nursing task uploaded by the medical end after the home care task is completed; on the other hand, after the home care task is completed, the home care evaluation data uploaded by the user end is obtained. The nursing operation video clips of each service item are accurately obtained from the nursing task audio and video data by clustering processing. The authenticity of the home care evaluation data is verified and adjusted by the graphic similarity between the home care evaluation data and the nursing operation video clips, and the real quality evaluation data of the home care task is obtained, which can timely identify the medical staff's forced good reviews and the user's malicious bad reviews, and reasonably protect the rights and interests of medical staff and users.
[0083] In some embodiments, see Figure 2 , clustering processing is performed according to the audio and video fusion features to intercept at least one nursing operation video clip from the nursing task audio and video data, the steps of which include:
[0084] S210: Cluster the audio and video fusion features to obtain a target feature set of the service item.
[0085] The target feature set corresponds to the project operation time period. For example, after the medical staff arrives at the user's home, they turn on the audio and video recording function on the medical terminal 130 in the preparation stage before the work starts. When a nursing service project starts, they click start on the medical terminal 130, and when the nursing ends, they click end on the medical terminal 130. In this way, the medical terminal 130 can record which operation time period corresponds to which nursing service project, and the data can also be uploaded to the platform management terminal 120.
[0086] Among them, when different nursing service items are carried out, the spatial characteristics collected by images, the nursing action characteristics of medical staff, the action characteristics of the cared persons, the cooperation action characteristics of family members, etc. are all different. The content characteristics of the medical staff's explanations collected by audio and the content characteristics of the conversations with the cared persons or family members are also different. The superposition of these factors can make the audio and video fusion characteristics of different nursing service items different. Therefore, clustering processing can be used to obtain the target feature sets of different service items.
[0087] S220, extracting the nursing operation video clip corresponding to the project operation time period from the nursing task audio and video data.
[0088] Specifically, according to the target feature set of any nursing service item and the time period data of the operation of the item, the video clip of the service item can be intercepted from the audio and video data of the nursing task. Because in the original audio and video, even during the occurrence of the same nursing item, there may be non-nursing service content, such as rest and chatting in the middle, which need to be filtered out, so the nursing operation video clips of any nursing service item intercepted may be one or more.
[0089] In the above embodiment, a target feature set of the service item is obtained based on clustering processing, and a nursing operation video clip is captured based on the target feature set of the service item and the operation time period data, thereby achieving accurate capture of the nursing item operation video clip, and providing more accurate video data for objective evaluation of nursing services.
[0090] In some embodiments, clustering the audio and video fusion features to obtain a target feature set of the service item includes the following steps:
[0091] S310: Clustering the audio and video fusion features to obtain multiple fusion feature sets.
[0092] Among them, the audio and video data corresponding to the fusion features included in the fusion feature set belong to the same place of occurrence. In some embodiments, a nursing service includes multiple nursing projects, and different nursing projects are carried out in different locations. The medical staff clicks start on the medical terminal 130 before each nursing project starts, and clicks end on the medical terminal 130 when the nursing project ends. After the recorded full audio and video files and the nursing time period data corresponding to each project are uploaded to the platform management terminal 120, the platform management terminal 120 intercepts the audio and video files according to the nursing time period data to obtain the audio and video files of each nursing project. For example, if there are 3 nursing projects in a nursing service order of a certain user, then the platform management terminal 120 will intercept 3 audio and video files, and the platform management terminal 120 can name them with the name of the nursing project when saving. It can be understood that the platform management terminal 120 clusters the audio and video fusion features contained in the intercepted audio and video files to obtain multiple fusion feature sets, including the fusion feature sets of multiple nursing projects, and may also include some fusion feature sets of rest or chat scenes.
[0093] S320 , screening multiple fused feature sets according to the image data of the location where the service project occurs, to obtain a target feature set of the service project.
[0094] In some embodiments, the features of the image data of the occurrence location may be extracted to obtain feature A, and feature A is used to calculate the similarity with the cluster center features of each fused feature set. The fused feature set corresponding to the highest similarity is the target feature set of the service item. In other embodiments, the features of the image data of the occurrence location may be extracted to obtain feature A, and each fused feature set is processed with equal weights and then similarity is calculated with feature A. The fused feature set corresponding to the highest similarity is the target feature set of the service item.
[0095] In some embodiments, before performing clustering processing according to the audio and video fusion features to intercept at least one nursing operation video clip from the nursing task audio and video data, the method may also include: obtaining home care demand data uploaded by the user terminal before the home care task is started. The home care demand data includes image data of the location where the service items occur; different service items correspond to different locations.
[0096] Specifically, after the user terminal 110 submits the home care demand data, the platform management terminal 120 obtains the data. After the medical staff responds to the order, the platform management terminal 120 generates the evaluation information of the nursing demand based on the nursing item data in the nursing demand data and the evaluation configuration data of each nursing item in the system. Furthermore, the home care demand also needs to include the image data of the location where the service item occurs, so that the medical staff can determine whether the user's home environment meets the needs of home care services. It can also be used for subsequent feature comparison to intercept video clips of different nursing items.
[0097] It is understandable that different service projects have different requirements for the environment, so they correspond to different locations. For example, the nursing environment required for the baby massage project may be on a flat bed with a baby pillow. For another example, the environment required for the maternity rehabilitation exercise project may be an open space of about 3 square meters with a yoga mat.
[0098] In some embodiments, in home care services, the safety of both medical staff and users is of vital importance. On the one hand, it is necessary to ensure that medical staff arrive at the correct nursing service location, which means that the whereabouts of medical staff can be tracked and determined to ensure the safety of medical staff; on the other hand, it is necessary to ensure that the medical staff who accepts the order come to the user's home, and are not used by people with ulterior motives to enter the user's home, so as to ensure the safety of the user and his family. To achieve this goal, this embodiment requires both parties to provide necessary information to the platform management end 120 for verification. Therefore, the home care demand data also includes the first geographic location information of the service item, the first porch image data, and the designated medical staff identification. The first geographic location information can be the GPS information of the user's home, the first porch image data can be the image of the entrance door taken at the porch position, and the designated medical staff identification can be the name or other identity information of the designated medical staff. The medical staff may have submitted identity authentication information when registering the home care intelligent management system and performed face recognition, so the platform management end 120 stores the identification feature correspondence between the medical staff identification and the medical staff face feature.
[0099] For further information, see Figure 3 The intelligent home care management method in this embodiment also includes the following steps:
[0100] S410, searching in the identification feature correspondence relationship according to the designated medical staff identification to obtain the target facial features.
[0101] Specifically, when the nursing demand information submitted by the user includes a designated medical staff identifier, after the platform management end 120 obtains the information, it can use the designated medical staff identifier to search in the identifier feature correspondence to obtain the target facial features.
[0102] S420, obtaining the second geographic location information, the second porch image data and the facial features of the medical staff uploaded by the medical staff end.
[0103] The second porch image data may be a photo of the user's entrance door taken by the medical staff when they arrive at the user's porch, the second geographic location information may be the GPS information acquired by the medical terminal 130 when the medical staff arrive at the user's porch, and the medical facial features may be the facial features acquired by the medical terminal 130 when the medical staff uploads the second porch image data and the medical terminal 130 requires facial verification. It is understandable that when uploading the second porch image data, the medical terminal 130 may automatically acquire the GPS information for uploading.
[0104] S430. If the second geographic location information is consistent with the first geographic location information, and the similarity between the target facial features and the medical staff facial features exceeds the similarity threshold, the service address is reviewed based on the second porch image data and the first porch image data. If the review is passed, an arrival reminder message is sent to the user terminal to remind the medical staff to arrive at the execution location of the service project.
[0105] It is understandable that the use of geographic location information plus porch image data can more effectively determine the user's location, and facial feature comparison can accurately identify the identity of medical staff. When all of this information is verified, the user can be reminded that the medical staff has arrived, and can also be reminded that the medical staff has arrived at the correct user location.
[0106] In the above embodiment, the security of home care services is ensured by comprehensively using the geographic location information and porch image data of the user's home and the facial feature verification of the medical staff. On the one hand, the real-time tracking of the geographic location and porch image is used to ensure that the medical staff has arrived at the correct user location. On the other hand, the identity of the medical staff is verified through facial recognition technology to prevent identity fraud, thereby achieving double security protection for users and medical staff.
[0107] In some embodiments, there are multiple service items; the evaluation text features include the item evaluation features of each service item; the home care evaluation data include the item evaluation data of each service item; and the evaluation text features of the nursing evaluation text data and the operation video features of the nursing operation video clip are matched to obtain the nursing evaluation weight factor. The method includes the following steps:
[0108] S510: For any service item, perform similarity calculation on the project evaluation feature of any service item and the operation video feature of any service item to obtain the image and text similarity data of any service item.
[0109] Specifically, feature extraction can be performed based on the evaluation text of any service project to obtain the project evaluation feature of the nursing service project, feature extraction can be performed on the intercepted audio and video files of the nursing project to obtain the operation video feature of the nursing project, and the image-text similarity data of the nursing service project can be determined based on the project evaluation feature and the operation video feature of the nursing service project. Exemplarily, the cosine similarity or Euclidean distance of the two features is calculated to obtain the corresponding image-text similarity data.
[0110] S520, determining a nursing evaluation weight factor for any service item based on the graphic and text similarity data of any service item.
[0111] Specifically, the graphic and text similarity data of any service item can be directly regarded as the nursing evaluation weight factor of any service item. It is also possible to pre-construct a mapping relationship between graphic and text similarity data and weight factors, and determine the nursing evaluation weight factor based on the mapping relationship. Exemplarily, the graphic and text similarity data and the nursing evaluation weight factor of any service item can be a one-to-one mapping relationship, or other corresponding relationships. The nursing evaluation weight factor corresponding to the graphic and text similarity data of any service item is determined based on the corresponding relationship.
[0112] The home care evaluation data is adjusted according to the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task, including:
[0113] S530. Adjust the project evaluation data of any service project based on the nursing evaluation weight factor of any service project to obtain adjusted evaluation data of any service project.
[0114] Specifically, the home care evaluation data of any service item is readjusted using the nursing evaluation weight factor of the service item to obtain the true quality evaluation data of the home care task of any service item.
[0115] S540. Integrate the adjusted evaluation data of any service item to obtain the real quality evaluation data of the home care task.
[0116] Specifically, when there are multiple service items in a service task, there are multiple ways to integrate the adjusted evaluation data of different service items. For example, the adjusted scores of each item can be added directly, or they can be added according to their respective weights. The weights of each item can be equal or unequal.
[0117] In some embodiments, a multimodal nursing quality evaluation model is deployed in the platform management end, and the multimodal nursing quality evaluation model includes a text encoder and an image encoder obtained by joint training; before performing feature matching based on the evaluation text features of the nursing evaluation text data and the operation video features of the nursing operation video clip to obtain the nursing evaluation weight factor, the method may further include:
[0118] S610: Input the nursing evaluation text data and the nursing operation video clips into the multimodal nursing quality evaluation model.
[0119] S620. Perform feature extraction on the nursing evaluation text data through a text encoder to obtain evaluation text features.
[0120] S630: extract features from the nursing operation video clips through an image encoder to obtain operation video features.
[0121] In this embodiment, in order to determine the nursing operation video clip with the help of video understanding technology, the nursing evaluation text data and the nursing operation video clip are first determined; secondly, a multimodal nursing quality evaluation model is constructed by jointly training the text encoder and the image encoder; then, the text encoder is used to extract features of the nursing evaluation text data to obtain evaluation text features, and the image encoder is used to extract features of the nursing operation video clip to obtain operation video features, so that the authenticity of the home care evaluation data can be verified and adjusted in combination with the graphic similarity between the home care evaluation data and the nursing operation video clip.
[0122] In some embodiments, feature extraction and feature fusion are performed based on the nursing task audio and video data to obtain audio and video fusion features, including:
[0123] S710 . Generate subtitles for process video data of the home care task based on process audio data of the home care task.
[0124] Specifically, you can first use speech recognition software to convert the audio into text, and then use subtitle production tools (such as Aegisub, Subtitle Edit, etc.) to align the text with the timeline in the video to ensure that the timing of the subtitles matches the audio. Finally, use video editing software (such as Adobe Premiere, Final Cut Pro, etc.) to embed the subtitle file into the video to generate a video file with subtitles.
[0125] S720: Generate image-caption pairing data based on the captions and the process video data.
[0126] The process video data includes a series of image frames, and the corresponding subtitles are on each image frame, so that each image frame and the corresponding subtitles constitute a set of image-subtitle pairing data.
[0127] S730 , extracting and fusing features of the image frames and subtitle texts in the image-subtitle pairing data to obtain audio-video fusion features.
[0128] The audio and video fusion features are used to represent the overall features of the image frame and the corresponding subtitle text in the image-subtitle pairing. The audio and video fusion features can be represented by vectors or in other forms. The audio and video fusion features can be directly obtained by concatenating the image frame features and the subtitle text features, or by extracting the image frame features and the subtitle text features based on the cross-attention mechanism.
[0129] Specifically, the image frames in the image-subtitle pairing are encoded to obtain image features of the image frames, and the subtitle texts corresponding to the image frames are encoded to obtain text features of the subtitle texts. Encoding can be performed based on the image features of the image frames and the text features of the subtitle texts to obtain fusion features of the image-subtitle pairing.
[0130] In the embodiment of the present application, a home care intelligent management device 800 is also provided, which is applied to the platform management end 120 of the home care intelligent management system. The home care intelligent management system also includes a user end 110 and a medical end 130 that are communicatively connected to the platform management end 120; the medical end 130 uploads the image data of the location where the service project occurs to the platform management end 120 before the home care task is started; the medical end 130 uploads the audio and video data of the care task to the platform management end 120 after the home care task is completed; the home care task includes at least one service item; please refer to Figure 4 The device includes a feature extraction and fusion module 810, an operation video acquisition module 820, a text video matching module 830, an evaluation category determination module 840, and an evaluation data adjustment module 850.
[0131] A feature extraction and fusion module 810 is used to extract and fuse features based on the audio and video data of the nursing task to obtain audio and video fusion features;
[0132] The operation video acquisition module 820 is used to perform clustering processing according to the audio and video fusion features to obtain multiple fusion feature sets; and to intercept the nursing operation video clip of at least one service item from the nursing task audio and video data according to the image data of the location where the service item occurs and the multiple fusion feature sets;
[0133] The text-video matching module 830 is used to perform feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip;
[0134] An evaluation category determination module 840 is used to determine the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor; wherein the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review;
[0135] The evaluation data adjustment module 850 is used to adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task.
[0136] In some embodiments, the evaluation category determination module 840 is also used to determine that the home care evaluation data belongs to the malicious bad review category if the nursing evaluation weight factor is less than the preset adjustment threshold and the difference between the total evaluation score and the home care evaluation data is continued to be determined to be less than the second preset difference threshold, and the home care evaluation data belongs to the forced good review category if the nursing evaluation weight factor is less than the preset adjustment threshold and the difference between the total evaluation score and the home care evaluation data is continued to be determined to be less than the second preset difference threshold; wherein the second preset difference threshold is greater than the first preset difference threshold.
[0137] In some embodiments, the home care demand data also includes a first geographic location information of the service item, a first porch image data, and a designated medical staff identification; the platform management terminal 120 stores an identification feature correspondence between the medical staff identification and the medical staff facial features; the device also includes a security audit module, which is used to search the identification feature correspondence according to the designated medical staff identification to obtain the target facial features; obtain the second geographic location information, second porch image data and medical staff facial features uploaded by the medical terminal 130 before the home care task is started; if the second geographic location information is consistent with the first geographic location information, and the similarity between the target facial features and the medical staff facial features exceeds the similarity threshold, the service address is audited according to the second porch image data and the first porch image data, and if the audit is passed, an arrival reminder message is sent to the user terminal 110 to remind the medical staff to arrive at the execution site of the service item.
[0138] In the embodiment of the present application, a home care intelligent management system is also provided. Please continue to refer to 1a. The home care intelligent management system includes a platform management terminal 120, a user terminal 110 and a medical terminal 130 which are connected to the platform management terminal 120 in communication; wherein:
[0139] The user terminal 110 is used to upload the image data of the location of the service project to the platform management terminal 120 before the home care task is started, and to upload the home care evaluation data to the platform management terminal 120 after the home care task is completed; the home care task includes at least one service project;
[0140] The medical terminal 130 is used to upload the audio and video data of the nursing task to the platform management terminal 120 after the home nursing task is completed;
[0141] The platform management terminal 120 is used to perform feature extraction and feature fusion based on the audio and video data of the nursing task to obtain audio and video fusion features; perform clustering processing based on the audio and video fusion features to obtain multiple fusion feature sets; and based on the image data of the location where the service project occurs and the multiple fusion feature sets, intercept at least one nursing operation video clip of the service project from the audio and video data of the nursing task; perform feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; determine the evaluation category of the home care evaluation data based on the difference between the total evaluation score of the home care task and the home care evaluation data and the nursing evaluation weight factor; adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip; the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review.
[0142] In some implementations, the platform management terminal 120 stores the identification feature correspondence between the medical staff identification and the medical staff facial features;
[0143] The user terminal 110 is also used to upload home care demand data to the platform management terminal 120 before the home care task is started; the home care demand data includes the image data of the location where the service project occurs, the first geographical location information of the service project, the first porch image data, and the designated medical staff identification;
[0144] The medical terminal 130 is also used to upload the second geographical location information, the second porch image data and the medical staff face features to the platform management terminal 120 before the home care task is started;
[0145] The platform management end 120 is also used to search in the identification feature correspondence relationship according to the designated medical staff identification to obtain the target facial features; if the second geographic location information is consistent with the first geographic location information, and the similarity between the target facial features and the medical staff facial features exceeds the similarity threshold, the service address is reviewed based on the second porch image data and the first porch image data. If the review is passed, an arrival reminder message is sent to the user end 110 to remind the medical staff to arrive at the execution location of the service project.
[0146] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0147] See also Figure 5 , Figure 5 It is a structural diagram of a computer device provided by an embodiment of the present application, and the computer device includes: one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example. The computer device further comprises a communication interface 30, which is used for the computer device to communicate with other devices or communication networks.
[0148] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0149] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.
[0150] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations all fall within the scope defined by the appended claims. For the convenience of description, the above devices are described separately by function in various units. Of course, when implementing the present application, the functions of each unit can be implemented in the same or more software and / or hardware. The above description is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of the claims of the present application.
[0151] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A home care intelligent management method, characterized in that: A platform management end applied to a home care intelligent management system, the home care intelligent management system further comprising a user end and a medical end connected to the platform management end in communication; the user end uploads image data of a location where a service item occurs to the platform management end before a home care task is started; the medical end uploads audio and video data of the care task to the platform management end after the home care task is completed; The home care task includes at least one service item; the method includes: Perform feature extraction and feature fusion based on the audio and video data of the nursing task to obtain audio and video fusion features; Performing clustering processing according to the audio and video fusion features to obtain multiple fusion feature sets; and intercepting a nursing operation video clip of at least one service item from the nursing task audio and video data according to the image data of the location where the service item occurs and the multiple fusion feature sets; Feature matching is performed based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip; Determining the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor; wherein the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review; The home care evaluation data is adjusted according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain true quality evaluation data of the home care task.
2. The method according to claim 1, characterized in that: The audio and video data corresponding to the fusion features included in the fusion feature set belong to the same occurrence location; the video clip of the nursing operation of at least one service item is intercepted from the nursing task audio and video data according to the image data of the occurrence location of the service item and the multiple fusion feature sets, including: The plurality of fused feature sets are screened according to the image data of the location where the service project occurs, to obtain a target feature set of the service project; wherein the target feature set corresponds to a project operation time period; A nursing operation video clip corresponding to the project operation time period is intercepted from the nursing task audio and video data.
3. The method according to claim 2, characterized in that Before intercepting at least one nursing operation video clip from the nursing task audio and video data, the method further includes: Acquire the home care demand data uploaded by the user terminal before the home care task is started; wherein the home care demand data includes image data of the occurrence location of the service item; different service items correspond to different occurrence locations.
4. The method according to claim 3, characterized in that The home care demand data also includes the first geographic location information of the service item, the first porch image data, and the designated medical staff identification; the platform management terminal stores the identification feature correspondence between the medical staff identification and the medical staff facial features; The method further comprises: According to the designated medical staff identification, the identification feature correspondence relationship is searched to obtain the target facial features; Acquire the second geographic location information, the second porch image data and the facial features of the medical staff uploaded by the medical staff before the home care task is started; If the second geographic location information is consistent with the first geographic location information, and the similarity between the target facial features and the medical staff facial features exceeds the similarity threshold, the service address is reviewed based on the second porch image data and the first porch image data. If the review is passed, an arrival reminder message is sent to the user terminal to remind the medical staff to arrive at the execution location of the service project.
5. The method according to claim 1, characterized in that The number of the service items is multiple; the evaluation text features include the project evaluation features of each service item; The home care evaluation data includes project evaluation data for each service project; The evaluation text features based on the home care evaluation data and the operation video features of the nursing operation video clips are matched to obtain a nursing evaluation weight factor; including: For any service item, similarity calculation is performed between the item evaluation feature of the any service item and the operation video feature of the any service item to obtain the image and text similarity data of the any service item; Determine a nursing evaluation weight factor for any of the service items based on graphic and text similarity data for any of the service items; wherein, adjust the project evaluation data for any of the service items by using the nursing evaluation weight factor for any of the service items to obtain adjusted evaluation data for any of the service items; integrate the adjusted evaluation data for any of the service items to obtain true quality evaluation data for the home care task.
6. The method according to claim 1, characterized in that The extracting and fusing features based on the nursing task audio and video data to obtain audio and video fusion features includes: generating subtitles for the process video data of the home care task based on the process audio data of the home care task; generating image-caption pairing data based on the caption and the process video data; Feature extraction and feature fusion are performed on the image frames and subtitle texts in the image-subtitle pairing data to obtain the audio-video fusion feature.
7. The method according to any one of claims 1 to 6, characterized in that: The step of determining the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor comprises: If the nursing evaluation weight factor is less than the preset adjustment threshold, and it is further determined that the difference between the total evaluation score and the home care evaluation data exceeds the first preset difference threshold, it is determined that the home care evaluation data belongs to the malicious bad review category; If the nursing evaluation weight factor is less than the preset adjustment threshold, continue to determine that the difference between the total evaluation score and the home care evaluation data is less than the second preset difference threshold, and determine that the home care evaluation data belongs to the forced positive review category; wherein the second preset difference threshold is greater than the first preset difference threshold.
8. An intelligent home care management system, characterized in that: The home care intelligent management system includes a platform management end, a user end and a medical end connected to the platform management end in communication; wherein: The user terminal is used to upload the image data of the location of the service project to the platform management terminal before the home care task is started, and to upload the home care evaluation data to the platform management terminal after the home care task is completed; the home care task includes at least one service project; The medical end is used to upload the audio and video data of the nursing task to the platform management end after the home nursing task is completed; The platform management end is used to perform feature extraction and feature fusion based on the audio and video data of the nursing task to obtain audio and video fusion features; perform clustering processing based on the audio and video fusion features to obtain multiple fusion feature sets; and based on the image data of the location where the service project occurs and the multiple fusion feature sets, intercept at least one nursing operation video clip of the service project from the audio and video data of the nursing task; perform feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip to obtain a nursing evaluation weight factor; determine the evaluation category of the home care evaluation data based on the difference between the total evaluation score of the home care task and the home care evaluation data and the nursing evaluation weight factor; adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain the real quality evaluation data of the home care task; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip; the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review.
9. The system according to claim 8, characterized in that The platform management terminal stores the identification feature correspondence between the medical staff identification and the medical staff facial features; The user terminal is also used to upload home care demand data to the platform management terminal before the home care task is started; the home care demand data includes image data of the location where the service item occurs, first geographical location information of the service item, first porch image data, and designated medical staff identification; The medical care end is also used to upload the second geographical location information, the second porch image data and the medical care face features to the platform management end before the home care task is started; The platform management end is also used to search the corresponding relationship of the identification features according to the designated medical staff identification to obtain the target facial features; if the second geographic location information is consistent with the first geographic location information, and the similarity between the target facial features and the medical staff facial features exceeds a similarity threshold, a service address review is performed based on the second porch image data and the first porch image data. If the review is passed, an arrival reminder message is sent to the user end to remind the medical staff to arrive at the execution site of the service project.
10. An intelligent home care management device, characterized in that: A platform management end applied to a home care intelligent management system, the home care intelligent management system further comprising a user end and a medical end connected to the platform management end in communication; the user end uploads image data of a location where a service item occurs to the platform management end before a home care task is started; the medical end uploads audio and video data of the care task to the platform management end after the home care task is completed; The home care task includes at least one service item; the device includes: A feature extraction and fusion module is used to extract and fuse features based on the audio and video data of the nursing task to obtain audio and video fusion features; an operation video acquisition module, configured to perform clustering processing according to the audio and video fusion features to obtain a plurality of fusion feature sets; and to intercept a nursing operation video clip of at least one service item from the nursing task audio and video data according to the image data of the location where the service item occurs and the plurality of fusion feature sets; A text-video matching module, used for performing feature matching based on the evaluation text features of the home care evaluation data and the operation video features of the nursing operation video clip, to obtain a nursing evaluation weight factor; wherein the nursing evaluation weight factor is used to characterize the consistency between the home care evaluation data and the nursing operation video clip; An evaluation category determination module, used to determine the evaluation category of the home care evaluation data according to the difference between the total evaluation score of the home care task and the home care evaluation data and the care evaluation weight factor; wherein the evaluation category is used to describe whether the home care evaluation data is a forced positive review or a malicious negative review; An evaluation data adjustment module is used to adjust the home care evaluation data according to the evaluation category, the total evaluation score, and the nursing evaluation weight factor to obtain true quality evaluation data of the home care task.
Citation Information
Patent Citations
Online consultation system and method
CN113096824A
Garbage classification intelligent supervision tracking evaluation management system
CN118071211A
The method for promoting a unknown music on internetusing play room and promotion system
KR1020030000185A
KR20210027915A
