Standardized patient auxiliary evaluation method for clinical practice ability assessment of physicians

Abstract

The invention discloses a standardized patient auxiliary evaluation method for clinical practice ability assessment of doctors, and belongs to the technical field of medical education and assessment, and the method comprises the following steps: 1, selecting examinees and examiners participating in the standardized training clinical practice ability check-out assessment of resident doctors as research objects; 2, the investigation step comprises the steps of clinical skills, clinical thinking, clinical communication and nervous system examination, synchronous and independent scoring is carried out by examiners subjected to unified training, and an average score is taken as an examination result; through a four-site examination mode of clinical skills, thinking, communication and nervous system examination, the clinical practice ability of the resident physician can be comprehensively evaluated, the application of the SP makes the examination more real, and the improvement of the clinical practice skills and communication ability of the resident physician is facilitated. Through questionnaire survey and data analysis, feedback in the assessment process can be collected, and a basis is provided for improving the assessment method and improving the application effect of the SP.

Description

Technical Field

[0001] The present invention relates to the field of medical education and evaluation technology, and in particular to a standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians. Background Art

[0002] Since Professors Barrows and Abrahamson proposed the concept of standardized patients in 1964, SP has been widely used in skill training, assessment and cultivation in medical education in many countries. The so-called SP refers to a person who, after systematic training, can accurately and realistically play the clinical symptoms and signs of patients, and act as a patient, evaluator and feedback provider. The object can be a normal person or the patient himself. Therefore, it is extremely important to cultivate practical talents in mental illness and mental health that meet social needs. The core of its cultivation is to build clinical thinking and clinical practice ability. Clinical practice is also the continuation and improvement of theoretical knowledge.

[0003] The appearance, voice tone, body movements, consistency and language accuracy of the SP are important factors that affect the experience of candidates participating in skill training, and the use of medical terminology by the SP is an important observation point for the examiner. This may be related to the candidate's mentality before the test, such as the urgency to obtain diagnostic support information from the SP's medical history description, the candidate's contact time with the SP is shorter than the examiner, and the examiner himself may also play the role of a family member. In addition, people with a medical background are prone to bring in medical terminology. For this reason, a standardized patient-assisted evaluation method for the assessment of physician clinical practice ability is proposed. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one purpose of the present invention is to propose a standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians includes the following steps:

[0007] Step 1: Select candidates and examiners who participated in the final examination of clinical practice ability of standardized training for resident physicians as research subjects;

[0008] Step 2: The assessment steps are clinical skills, clinical thinking, clinical communication and neurological examination, which are scored simultaneously and independently by examiners who have received unified training, and the average score is taken as the assessment result;

[0009] Step 3: After the assessment, an anonymous questionnaire survey was conducted to collect the examiners’ and candidates’ satisfaction evaluations on the assessment process and SP performance;

[0010] Step 4: The collected data were statistically analyzed using SPSS26.0 and GraphPad Prism 8.0, and the images were processed using Adobe Illustrator, and finally the evaluation conclusions were drawn.

[0011] As a further optimization scheme of the present invention, in step two, clinical skills are assessed on mental examination, history collection, and necessary physical examination; clinical thinking is assessed on case analysis, answering questions, and writing of medical records summaries; clinical communication is assessed on common communication topics for residents; and neurological examination is assessed on cranial nerves, motor system, sensory system, and pathological signs.

[0012] As a further optimization scheme of the present invention, in step three, the questionnaire includes positive and negative questions and adopts the Likert 5-level scoring system.

[0013] As a further optimization scheme of the present invention, in step 4, the examination scores are described as mean ± standard deviation, and the Kruskal-Wallis rank sum test is used for statistical analysis, and P < 0.05 indicates that the difference is statistically significant;

[0014] The answers to the questionnaire were converted into percentage descriptions, and the Kendall test was used to analyze the consistency of the answers to the questionnaires of the two groups. P < 0.05 indicated that the consistency was statistically significant. The Kendall coordination coefficient < 0.2 indicated that the consistency was poor, 0.2-0.4 indicated that the consistency was average, 0.4-0.6 indicated that the consistency was moderate, 0.6-0.8 indicated that the consistency was strong, and 0.8-1.0 indicated that the consistency was very strong.

[0015] The Wilcoxon rank-sum test was used to analyze the differences between the two groups of questionnaires, and P < 0.05 indicated that the difference was statistically significant.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] Through the four-site assessment method of clinical skills, thinking, communication and neurological examination, the clinical practice ability of residents can be comprehensively evaluated. Through anonymous questionnaire surveys and statistical analysis, the objectivity and fairness of the assessment are improved, ensuring the scientificity and reliability of the evaluation results. The application of SP makes the assessment more realistic and helps to improve the clinical practice skills and communication ability of residents. Through questionnaire surveys and data analysis, feedback during the assessment process can be collected to provide a basis for improving the assessment method and improving the application effect of SP. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1

[0020] The present invention provides a technical solution: a standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians, comprising the following steps:

[0021] Step 1: Select candidates and examiners who participated in the final examination of clinical practice ability of standardized training for psychiatric residents from 2020 to 2023 as the research subjects.

[0022] Step 2: Based on the 2020 assessment standards and in combination with the characteristics of psychiatric specialty, formulate a multi-site assessment format and establish an examination team to be responsible for the overall design, preparation and implementation of the assessment.

[0023] Assessment sites: There are four assessment sites in total, namely clinical skills, clinical thinking, clinical communication and neurological examination. Among them, the first, third and fourth sites involve medical students or doctors playing the role of patients or patients' families to simulate real medical scenarios.

[0024] Examiner scoring: Each assessment site is scored by two trained examiners from different professional bases and holds a completed practical skills examiner certificate. The examiners will score synchronously and independently, and the average score will be taken as the candidate's final score.

[0025] Assessment and grading system: All assessment items are scored on a percentage basis and are included in the total score after conversion based on the weight of each item, with a total of 100 points.

[0026] Table 1. Clinical practice skills final examination site information

[0027]

[0028] Standardized patient / family training: All medical students and doctors who play the role of patients or family members receive unified training to ensure that their performance in the assessment meets standardized requirements.

[0029] Content and implementation of each test station:

[0030] Clinical Skills Station:

[0031] Assessment content: Mental examination, medical history collection and necessary physical examination

[0032] Role-playing situation: The patient role in the psychiatric examination is played by medical students, while the doctor plays the role of the family member.

[0033] Clinical Thinking Station:

[0034] Assessment content: Involving case analysis, answering questions and medical record summary

[0035] Main contents: including symptom and syndrome analysis, diagnostic analysis, diagnostic conclusions and diagnostic criteria, risk assessment and treatment measures, and treatment plan.

[0036] Clinical Communication Station:

[0037] Assessment content: Candidates communicate with the examiner who plays the role of the patient's family member

[0038] Implementation method: The examiner plays the role of the patient's family member based on the characteristics of the relevant case and interacts and communicates with the examinee.

[0039] Nervous System Station:

[0040] Examination content: Candidates conduct specialized neurological examinations on medical students who act as patients. The examination assesses candidates’ professional skills and clinical operation capabilities in neurological examinations.

[0041] Step 3: After the assessment, an anonymous questionnaire survey was conducted to collect the examiners’ and candidates’ satisfaction evaluations on the assessment process and SP performance;

[0042] Specifically, the distribution and filling of questionnaires: On the day the assessment ends and before the test results are announced, anonymous questionnaires will be distributed to examiners and candidates through the online platform.

[0043] The content of the questionnaire covered the recognition of the examination work, examination content, difficulty, rationality of the format, and overall satisfaction of each examination center, with a special focus on the performance of standardized patients (SP).

[0044] The questionnaire design and scoring criteria include: Assessment satisfaction questionnaire: A 10-item questionnaire is designed for examiners and candidates respectively, aiming to collect satisfaction evaluations on various aspects of the assessment process.

[0045] SP satisfaction questionnaire: A 20-item questionnaire was designed, of which the first 10 items were positive questions and the last 10 items were negative questions. The items focused on evaluating the SP's performance in the assessment.

[0046] Scoring method: All questionnaire items were scored using the Likert 5-point rating system.

[0047] Scoring criteria: 5 points represent very satisfied / completely agree, 1 point represents very dissatisfied / completely disagree. Quantify the examiner's and candidate's satisfaction with the assessment and SP performance.

[0048] Step 4: The collected data were statistically analyzed using SPSS26.0 and GraphPad Prism 8.0, and the images were processed using Adobe Illustrator, and finally the evaluation conclusions were drawn.

[0049] Specifically, data collation and analysis: SPSS26.0 and GraphPad Prism 8.0 software were used to perform statistical analysis on the collected data. The score data of the test station were described in the form of mean ± standard deviation. The Kruskal-Wallis rank sum test was used to analyze the test station scores to determine whether there were statistically significant differences, where P < 0.05 indicated that the difference was statistically significant.

[0050] Analysis of questionnaire survey results: The questionnaire responses were converted into percentages. The Kendall test was used to analyze the consistency of the two groups' responses, where P < 0.05 indicated statistical significance.

[0051] The degree of consistency was assessed based on the Kendall coordination coefficient: less than 0.2 indicated poor consistency, 0.2-0.4 indicated fair consistency, 0.4-0.6 indicated moderate consistency, 0.6-0.8 indicated strong consistency, and 0.8-1.0 indicated very strong consistency.

[0052] The Wilcoxon rank-sum test was used to analyze the differences between the two groups, and P < 0.05 indicated statistically significant differences.

[0053] The practical conclusions obtained from steps one to four are as follows:

[0054] In the four years from 2020 to 2023, a total of 523 candidates participated in the clinical practice skills final assessment of standardized training for psychiatric residents. In terms of gender distribution, there were 187 male candidates and 336 female candidates. The overall average score of the candidates in the assessment was 83.88, with a standard deviation of 5.78.

[0055] Specifically for each assessment site, the average score for the first site was 83.77, with a standard deviation of 6.80;

[0056] The mean score for the second station was 81.50, with a standard deviation of 6.81;

[0057] The mean score for the third station was 84.46, with a standard deviation of 6.70;

[0058] The mean score for the fourth station was 88.31, with a standard deviation of 7.82.

[0059] According to the data in Table 2, except for the second station, the scores of other assessment stations showed significant differences over the years.

[0060] Table 2. Overall situation of practical skills final examination (mean ± standard deviation)

[0061]

[0062] Evaluation of examiners' and candidates' satisfaction with the final practical skills assessment

[0063] A total of 47 valid examiner questionnaires and 138 examinee questionnaires were collected, as shown in Tables 3 and 4. Those who answered “very satisfied” and “satisfied” were considered to be satisfied with the content of the items.

[0064] 89.36% of examiners were satisfied with the examination work as a whole, 85.51% of candidates were satisfied with the examination work as a whole, examiners' satisfaction with the examination environment and examination equipment was lower than the overall satisfaction (87.23%, 85.11% vs. 89.37%), and candidates' satisfaction with the examination environment, examination format, examination difficulty and scope of examination questions was lower than the overall satisfaction (78.99%, 83.33%, 78.99%, 84.06% vs. 85.51%).

[0065] Table 3. Examiners’ evaluation of satisfaction with the practical skills final examination (%)

[0066]

[0067] Table 4. Candidates’ evaluation of satisfaction with the practical skills final examination (%)

[0068]

[0069] Consistency of examiners' and candidates' satisfaction ratings on SP

[0070] The Kandall test was used to analyze the consistency of the examiners' and examinees' evaluation of satisfaction with SP. As shown in Table 5, the consistency of the evaluation was statistically significant, among which the consistency of the examiners was higher than that of the examinees. The Kandall coordination coefficient of the examiners was 0.829, indicating strong consistency, while the Kandall coordination coefficient of the examinees was 0.012, indicating weak consistency.

[0071] Table 5. Consistency test of standardized patient satisfaction evaluation

[0072]

[0073] Specific comments from examiners and candidates on SP

[0074] The SP evaluation questionnaire for examiners and candidates covers 20 items, the first 10 items are positive questions, and the last 10 items are negative questions, as shown in Table 6. The inter-group difference comparison shows that there are 7 items with different satisfaction levels between examiners and candidates (examiners vs. candidates, strongly agree + agree), namely: consistent with the patient's appearance (100% vs. 92.75%), the patient skillfully responds to the medical history information with appropriate voice and intonation (98.08% vs. 94.21%), consistent with the patient's body movements (98.08% vs. 93.48%), the patient's performance is consistent and coordinated (100% vs. 95.65), the patient accurately expresses language (96.16% vs. 94.21%), the patient accurately expresses the psychological condition through language (98.07% vs. 96.37%) and the patient uses medical terms (17.31% vs. 0%). Among the 7 items, the first 6 were positive items and the 7th was a negative item. Although the satisfaction ratios showed that the examiners and candidates had the same trend, the examiners were more satisfied with the patient's appearance, voice tone, body movements, consistency and language accuracy than the candidates. On the use of medical terminology, the examiners were more likely than the candidates to think that SP used medical terminology.

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081] Parts not involved in the present invention are the same as the prior art or can be implemented by the prior art. Although the embodiments of the present invention have been shown and described, it is understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A standardized patient-assisted evaluation method for assessing physicians' clinical practice ability, characterized by: The following steps are involved: Step 1: Select candidates and examiners who participated in the final examination of clinical practice ability of standardized training for resident physicians as research subjects; Step 2: The assessment steps are clinical skills, clinical thinking, clinical communication and neurological examination, which are scored simultaneously and independently by examiners who have received unified training, and the average score is taken as the assessment result; Step 3: After the assessment, an anonymous questionnaire survey was conducted to collect the examiners’ and candidates’ satisfaction evaluations on the assessment process and SP performance; Step 4: The collected data were analyzed using SPSS26.0 and GraphPad Prism Statistical analysis was performed using Adobe Photoshop 8.0, and the images were processed using Adobe Illustrator to draw the final evaluation conclusions.

2. The standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians according to claim 1 is characterized by: In step two, clinical skills are assessed in terms of mental examination, history taking, and necessary physical examination; clinical thinking is assessed in terms of case analysis, answering questions, and writing of medical records summaries; clinical communication is assessed in terms of common communication topics for residents; and neurological examinations are assessed in terms of cranial nerves, motor system, sensory system, and pathological signs.

3. The standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians according to claim 1 is characterized by: In step three, the questionnaire included positive and negative questions and adopted a 5-point Likert rating system.

4. The standardized patient-assisted evaluation method for evaluating the clinical practice ability of physicians according to claim 1 is characterized by: In step 4, the examination scores were described as mean ± standard deviation, and statistical analysis was performed using the Kruskal-Wallis rank sum test, with P < 0.05 indicating statistically significant differences; The answers to the questionnaire were converted into percentage descriptions, and the Kendall test was used to analyze the consistency of the answers to the questionnaires of the two groups. P < 0.05 indicated that the consistency was statistically significant. The Kendall coordination coefficient < 0.2 indicated that the consistency was poor, 0.2-0.4 indicated that the consistency was average, 0.4-0.6 indicated that the consistency was moderate, 0.6-0.8 indicated that the consistency was strong, and 0.8-1.0 indicated that the consistency was very strong. The Wilcoxon rank-sum test was used to analyze the differences between the two groups of questionnaires, and P < 0.05 indicated that the difference was statistically significant.

Images