Newborn care evaluation method based on image and dry-wet color development fusion determination

Abstract

This invention discloses a neonatal care assessment method based on image and dry / wet colorimetric fusion judgment, belonging to the field of standardized neonatal care training and intelligent assessment technology. The method includes constructing a drying training carrier based on a neonatal simulator and a reversible water-induced color-changing film. Instructors use this carrier to teach drying techniques and obtain teaching standards. Training video data of trainees is acquired and matched in real-time with the teaching standards to obtain substandard operation data and substandard colorimetric data. The substandard operation data and substandard colorimetric data are then matched to determine their correspondence. Based on this correspondence, the trainee's correction data is determined, and an alarm signal is output based on the correction data. Trainee information is acquired, and the trainee's historical correction data is determined based on this information. After the trainee completes the current operation, an early warning is issued based on the historical correction data. This invention improves the accuracy of training assessment results.

Description

Technical Field

[0001] This application relates to the field of standardized training and intelligent assessment technology for neonatal care, and in particular to a neonatal care assessment method based on image and dry / wet colorimetric fusion. Background Technology

[0002] Early Essential Neonatal Care (EENC) is a core technology system for ensuring the life and health of newborns. Standardized drying procedures initiated immediately after birth are crucial for preventing hypothermia and stimulating spontaneous breathing. The timing, duration, sequence, and degree of dryness directly determine the intervention's effectiveness. Traditional training methods often rely on manual instruction, leading to a lack of quantifiable monitoring methods for time-series indicators during the teaching process. Training assessments depend on subjective recall and lack objective data support.

[0003] In related technologies, video data is often obtained by monitoring trainees' actions and then analyzing it to determine if there are any problems in newborn care. If a problem is identified, an alarm is triggered to standardize the trainees' actions. However, relying solely on video monitoring and judgment can only determine whether the trainee's actions meet the specified standards, but it cannot accurately reflect the trainee's actual performance. For example, a trainee's actions may meet the standards in the instructional video, but the actual dryness of the newborn may not meet the standards. In other words, the same action can lead to different results. Judging solely based on the actions may result in misjudgment. Therefore, it is necessary to also assess the results of the actions to ensure the accuracy of the training evaluation. This requires improvement. Summary of the Invention

[0004] To improve the accuracy of training assessment results, this application provides a neonatal care assessment method based on image and dry / wet colorimetric fusion determination.

[0005] This application provides a neonatal care assessment method based on the fusion of image and dry / wet colorimetric analysis, employing the following technical solution: Neonatal care assessment methods based on image and dry / wet colorimetric fusion include: Based on a newborn simulation body and a reversible water-induced color-changing film, a drying training carrier was constructed. Videos of the process of instructors using the drying training carrier to teach drying were collected to obtain teaching video data. Obtain behavioral tagging data of the instructor on the teaching video data, perform action analysis on the teaching video data based on the behavioral tagging data, determine the key events in the teaching video data and the key information of the key events, and obtain the first teaching operation data; Obtain colorimetric data from the teaching video data, and match the colorimetric data with the first teaching operation data to obtain the teaching standard; Acquire students' training video data and perform real-time operation matching and color matching with teaching standards to obtain substandard operation data and substandard color data; The substandard operation data is matched with the substandard color rendering data to determine the correspondence between the substandard operation data and the substandard color rendering data. Based on the correspondence, the corrective data of the trainee is determined, and an alarm signal is output based on the corrective data. Obtain student information and determine the student's historical correction data during the training process based on the student information. Perform a time-series judgment between the historical correction data and the student's current operation. If there is a time-series adjacency between the current operation and the historical correction data, provide an early warning to the student based on the historical correction data after the student completes the current operation.

[0006] Preferably, the teaching video data is split based on behavioral tagging data to obtain the teaching content under the corresponding tags; Based on the temporal order of the teaching content in the teaching video data, the operation timing data is obtained; The wiping actions under each marked teaching content are statistically analyzed in terms of path and coverage to obtain wiping action data; The duration of the wiping action is statistically analyzed to determine the duration of the wiping action. The data on wiping actions, duration, and sequence of operations are recorded as the first teaching operation data for neonatal drying care.

[0007] Preferably, based on the time sequence, the color development data is matched with the wiping action to determine the color development changes on the training carrier when the instructor performs the wiping action, and the color development change results are obtained; If the color development data of the covered area corresponding to each wiping action in the first teaching operation data changes in a positive direction, then the purpose of the wiping action is determined to be the wiping purpose. If the colorimetric data of the area covered by the wiping action in the first teaching operation data remains unchanged or changes in the opposite direction, then the purpose of the wiping action is determined to be a transitional purpose. Based on the behavioral purpose of each wiping action and the corresponding color change results of each wiping action, the first teaching operation data is marked with corresponding wiping actions to obtain the teaching standard.

[0008] Preferably, based on human dynamics, the movement line of the wiping action in the teaching video data is judged to determine the rationality of the wiping action movement line; If it is determined that the wiping action in the teaching video data does not have a reasonable movement path, then the wiping action is optimized according to the behavioral purpose of the wiping action to obtain the optimized behavior; The optimized behavior is fed back to the instructor, who then conducts a lesson based on the optimized behavior, resulting in a second optimized teaching result and second optimized colorimetric data. The second optimized colorimetric data is compared with the colorimetric data of the first teaching operation data, and the second optimized teaching results are compared with the optimized behaviors to determine the optimization effect; If the optimization effect is that the optimization behavior achieves better results, then the optimization behavior and the first optimized color data are selected as the teaching standard for subsequent teaching evaluation. If the optimization effect is unsatisfactory, the original teaching video data will be used as the teaching standard for subsequent teaching evaluation. If the optimization effect remains unchanged, then obtain feedback from the instructors and select teaching behaviors based on their feedback to obtain the final teaching standard.

[0009] Preferably, if there is a one-to-one correspondence between some substandard operation data and substandard color development data, then corrective data is obtained based on the key events corresponding to the substandard operation data; If some substandard operation data cannot be matched one-to-one with substandard color development data, then the non-matching substandard operation data will be recorded as pairing failure operation data, and the non-matching substandard color development data will be recorded as pairing failure color development data. If it is determined that there is pairing failure color data, then based on the time data of the pairing failure color data, it is determined whether there is pairing failure operation data in the time period before the time data of the pairing failure color data. If there is pairing failure operation data in the time period before the pairing failure color data, the key events corresponding to the pairing failure color data and the pairing failure operation data will be used as the student's correction data. If there is no pairing failure operation data in the time period before the pairing failure color data, the key event corresponding to the pairing failure color data will be used as the student's correction data. If it is determined that there is no pairing failure display data but only pairing failure operation data, then the pairing failure operation data will not be marked as the student's correction data.

[0010] Preferably, the second optimized teaching result is compared with the optimized behavior by wiping action to determine the similarity between the wiping action of the teacher and the wiping action in the optimized behavior, so as to obtain the execution perfection data; If the instructor's execution perfection data for the optimization behavior is greater than the built-in standard threshold, the color depth of the second optimization color data is compared with the color data of the first teaching operation data. If the color depth of the second optimized color data is lighter than the color data in the first teaching operation data, then the optimized behavior is judged to have a better wiping effect; If the color depth of the second optimized color data is deeper than that of the color data in the first teaching operation data, then the wiping effect of the optimization behavior is determined to be poor. If the color depth of the second optimized color data is the same as the color data in the first teaching operation data, then the wiping effect of the optimized behavior is determined to be the same. Obtain the execution time of the optimized behavior and compare it with the execution time in the first teaching operation data; If the execution time of the optimization behavior is longer, the optimization behavior is judged to be ineffective. If the execution time of the optimized behavior is equal to the execution time in the first teaching operation data, then the execution effect of the optimized behavior is determined to be unchanged; If the execution time of the optimization behavior is shorter, then the optimization behavior is considered to be more effective. When the wiping effect of the optimization behavior is better, and the execution effect is either better or unchanged, the optimization effect is determined to be better. When the wiping effect of the optimization behavior is better, but the execution effect is poor, an alarm signal for re-collection will be output. When the wiping effect of the optimization behavior is the same, and the execution effect is better, then the optimization effect is determined to be better; When the wiping effect of the optimization behavior is the same, but the execution effect is poor, the optimization effect is judged to be poor. When the wiping effect of the optimization behavior is the same, but the execution effect is unchanged, the optimization effect is determined to be unchanged. When the wiping effect of the optimization behavior is poor, but the execution effect is better, an alarm signal for re-collection will be output. When the wiping effect of the optimization behavior is poor, and the execution effect is poor or unchanged, the optimization effect is judged to be poor.

[0011] Preferably, the student's student information is obtained, and the corresponding student's training video data and its corresponding correction data are read based on the student's student information; Statistical analysis was performed on the correction data of the trainees to identify the frequently occurring correction data, which was then recorded as the correction data of the trainees to be analyzed. Based on the correction data of the students to be analyzed, determine the corresponding training video data and record it as the training data of the students to be analyzed; The wiping motion model of the student to be analyzed is obtained by learning the wiping motion from the student's training data. The wiping action model of the trainee to be analyzed is compared with the corresponding wiping action in the teaching standard to identify the differences; Based on human dynamics and distinguishing points, the wiping action model was simulated to determine how to make the wiping action model meet the wiping action requirements in the teaching standard by adjusting the limb movements while keeping the wiping action model unchanged, and thus obtain limb guidance data. Obtain the time point corresponding to the student's correction data to be analyzed, and obtain the wiping action before the corresponding time point to obtain the wiping action to be adjusted; Based on the body guidance data, the wiping action to be adjusted is adjusted to obtain a new teaching standard, and the training guidance and evaluation of students are based on the new teaching standard.

[0012] In summary, this application includes at least one of the following beneficial technical effects: 1. By utilizing a newborn simulation model and a reversible water-sensitive color-changing film, a drying training carrier was constructed. This integrated operational behavior video analysis with dry / wet color development result detection, avoiding misjudgments from single-behavior detection. By collecting data on the drying process during instruction using the training carrier, the rationality and authenticity of the instructional videos within the system were ensured. Furthermore, by acquiring behavioral marker data from the instructors' interactions with the instructional video data, the system further aided in understanding the instructors' drying behaviors, ensuring the accuracy of subsequent student training evaluations. Additionally, by combining color development data from the instructional process, a multi-faceted judgment method was adopted, moving away from a single criterion. The standards further improve the accuracy of evaluating trainee training results. By comparing trainees' training videos with the instructional videos in the teaching standards, it is possible to determine whether the trainees' operational behaviors and colorimetric results during training are correct. Then, substandard operational data is matched with substandard colorimetric data to further verify the trainees' errors, making the evaluation of trainee training more accurate. At the same time, by utilizing trainee information and statistically analyzing trainee training data, it is possible to use trainees' historical correction data to provide early warnings during subsequent training, thereby further improving the trainee training effect. 2. By leveraging the behavioral tagging data of instructors in instructional videos, the system can more accurately segment instructional videos and understand the instructors' operational behaviors within them. By utilizing the temporal sequence of behavioral tagging data within the instructional video data, the sequence of the drying care process is clarified. By statistically analyzing the path, coverage, and duration of wiping actions in the instructional content, the operational elements required for each step are further identified. This makes the final determined first instructional operation data more accurate, leading to more accurate subsequent instructional standards, more accurate analysis of student training videos, and more accurate evaluation of student training results, thereby improving the accuracy of newborn drying care judgments. 3. By matching substandard operational data with substandard color-coding data, the correspondence between them is determined. Substandard operational data with a one-to-one correspondence is marked as correction data, thus ensuring trainee training efficiency. Substandard operational data and substandard color-coding data that cannot be matched one-to-one are categorized, and color-coding data with failed pairings are analyzed to determine the possible causes. By matching failed pairing operational data prior to the corresponding failed pairing time point, correction data for failed pairings is determined, comprehensively covering the causes of substandard performance and improving trainee training effectiveness. Attached Figure Description

[0013] Figure 1 This is a flowchart of the steps in the neonatal care assessment method based on image and dry / wet colorimetric fusion judgment in this embodiment. Detailed Implementation

[0014] The following is in conjunction with the appendix Figure 1 This application will be described in further detail.

[0015] This application discloses a method for neonatal care assessment based on the fusion of image and dry / wet color development.

[0016] Example: Figure 1 As shown, the newborn care assessment method based on image and dry / wet colorimetric fusion judgment of the present invention includes: S1. Based on a newborn simulation body and a reversible water-induced color-changing film, a drying training carrier is constructed. Videos of the process of the instructor using the drying training carrier to teach drying are collected to obtain teaching video data. S2, acquire the instructor's behavioral marker data on the teaching video data, perform action analysis on the teaching video data based on the behavioral marker data, determine the key events in the teaching video data and the key information of the key events, and obtain the first teaching operation data; wherein, the behavioral marker data refers to the time node of the key event; for example, the time node of the key event such as starting to dry, ending to dry, and entering the warming / wrapping / skin contact. The key information includes the instructor's operation behavior in the corresponding key event.

[0017] S3, acquire color data from the teaching video data, and match the color data with the first teaching operation data to obtain the teaching standard; S4: Acquire the trainees' training video data and perform real-time operation matching and color matching with the teaching standards to obtain substandard operation data and substandard color data. When evaluating trainees' training results, the initial assessment is result-oriented, determining whether the water on the training medium was dried within the specified time. If the task was completed within the specified time, the operation was deemed correct. If the task was not completed within the specified time, time was traced back to determine if there were any issues with the duration of each drying action, thereby identifying substandard operational data. If the task was completed within the specified time, but the drying result did not meet the standard, the training video was compared with the instructional video in terms of wiping actions and color development results, thereby identifying substandard operational and color development data.

[0018] S5, match the substandard operation data with the substandard color display data, determine the correspondence between the substandard operation data and the substandard color display data, determine the corrective data of the trainee based on the correspondence, and output an alarm signal based on the corrective data; S6. Obtain the student's information and determine the student's historical correction data during the historical training process based on the student's information. Perform a time sequence judgment between the historical correction data and the student's current operation. If there is a time sequence adjacency between the current operation and the historical correction data, then provide an early warning prompt to the student based on the historical correction data after the student completes the current operation.

[0019] In this embodiment, a drying training carrier is constructed using a newborn simulator and a reversible water-sensitive color-changing film. This integrates operational behavior video analysis with dry / wet color development result detection, avoiding misjudgments from single-behavior detection. By collecting data on the drying process during instruction using the training carrier, the rationality and authenticity of the teaching videos within the system are ensured. Furthermore, by acquiring behavioral marker data from the instructors' interactions with the teaching video data, the system further understands the instructors' drying behaviors, ensuring the accuracy of subsequent student training evaluations. Additionally, by combining color development data from the teaching process, the system shifts from a single judgment criterion to a multi-faceted approach. The established evaluation criteria further improve the accuracy of student training outcome assessment. By comparing students' training videos with the instructional videos in the teaching standards, it determines whether students' operational behaviors and colorimetric results are correct during training. Furthermore, matching substandard operational data with substandard colorimetric data further verifies students' errors, making student training assessment more accurate. Simultaneously, by utilizing student information and statistically analyzing their training data, it enables the use of historical correction data to provide early warnings for students during subsequent training, further improving training effectiveness.

[0020] For example, by covering a newborn simulator with a reversible hydrochromic film, the system can reflect in real time whether the trainee has dried the simulator during newborn drying training. For instance, if area A is not dried, it displays a color saturation of 100. When the trainee dries area A, if area A is dry, its color saturation is 0; if the moisture in area A decreases but it is not completely dried, its color saturation is greater than 0 but less than 100. The color change of the reversible hydrochromic film reflects the amount of moisture remaining on the newborn simulator.

[0021] The instructor simulates the drying process using the drying training device and then captures the simulation video to obtain an instructional video. Since the instructional video is obtained through the instructor's simulation, it has certain guiding significance and can be used as reference data to guide students in drying training.

[0022] Meanwhile, since different sequences and methods of drying training may affect newborns, and different trainees have different operating habits, it is necessary to break down the teaching videos to identify the key information of the instructors in the teaching process. This will allow the drying behavior assessment to be conducted with the behavioral objectives in mind, thereby ensuring the accuracy of the assessment.

[0023] After capturing video footage of the instructor's simulated drying actions, the instructor uses the collected video data to describe the behavior (behavioral labeling data), determining the steps and procedures corresponding to each action in the video, thus manually breaking down the video. For example, if a 10-minute video contains steps a, b, and c, the instructor's labeling clarifies the time period corresponding to step a (e.g., 0-3 minutes), step b (3-6 minutes), and step c (6-9 minutes). This indicates that the 0-9 minute video content is used as an evaluation standard to assess the trainee's training results. This first instructional data serves as the trainee's action evaluation standard.

[0024] After establishing the evaluation criteria for trainees' movements, to prevent trainees from merely performing the actions without actually engaging in the drying process, it is also necessary to assess the color development of the training medium during the drying process. This assessment determines the amount of moisture loss resulting from each drying action, i.e., the color change. By combining the drying action with the color change, a corresponding teaching standard can be constructed, making the final teaching standard more effective and accurate.

[0025] After establishing the teaching standards for evaluating trainees' training progress, the system guides trainees in performing newborn drying care, ensuring they can successfully complete the entire process. The system also collects operation videos and color data during the process to determine the trainee's training status. For example, if a trainee is training without a teacher, they can follow the steps in the teaching standards. During this process, the system collects data on the trainee's actions and color changes on the drying surface to determine the training result. The training result is then compared with the teaching standards under overall guidance to identify any problems. In subsequent training sessions, because the system has stored the trainee's data, it can provide real-time matching and correction, and also provide early warnings based on past training issues, thus ensuring training effectiveness.

[0026] In step S2, behavioral tagging data of the instructor on the teaching video data is obtained. Based on the behavioral tagging data, action analysis is performed on the teaching video data to determine key events and key information of the key events in the teaching video data, thereby obtaining the first teaching operation data. This includes the following steps: S21, Based on the behavioral tagging data, the teaching video data is split to obtain the teaching content under the corresponding tags; S22, Based on the time sequence of the teaching content in the teaching video data, obtain the operation timing data; S23, perform path statistics and coverage statistics on the wiping actions of the teaching content under each mark to obtain wiping action data; S24, Perform duration statistics on the wiping action data to determine the duration data corresponding to the wiping action; S25, record the wiping action data, duration data, and operation sequence data as the first teaching operation data for newborn drying care.

[0027] In this embodiment, by utilizing the behavioral tagging data of instructors on teaching videos, the system can more accurately segment teaching videos and understand the instructors' operational behaviors within them. By using the temporal sequence of behavioral tagging data in the teaching video data, the process sequence of drying care is clarified. By statistically analyzing the path, coverage, and duration of wiping actions in the teaching content, the operational elements required for each step are further clarified, making the final determined first teaching operation data more accurate. This, in turn, makes the subsequently constructed teaching standards more accurate, the analysis of trainee training videos more accurate, and the evaluation of trainee training results more accurate, thereby improving the accuracy of newborn drying care judgment.

[0028] For example, when performing drying care on a newborn, the instructor's teaching video covers the entire drying process, which involves multiple steps, such as drying from the head to the torso and then to the limbs. To ensure the system's correct understanding, the instructor needs to mark the teaching video with behavioral tags, which helps the system understand the teaching video.

[0029] For example, in a 10-minute instructional video, the instructor uses markings to determine that 0-3 minutes represent head wiping actions, 3-6 minutes represent trunk wiping actions, 6-9 minutes represent limb wiping actions, and 9-10 minutes represent warming behaviors. The system categorizes the instructional video based on these markings and interprets them individually, ensuring both accuracy and speed in understanding the video.

[0030] After determining the video content under the corresponding tag based on the behavior tag data, the order between each step of the operation is further determined by combining the time sequence of the corresponding video content in the entire teaching video. For example, wipe the head first, then wipe the torso, and finally wipe the limbs. If there is any omission or random wiping, the wiping behavior can be judged as inaccurate and the evaluation result will not pass.

[0031] Furthermore, since a single wiping step involves multiple wiping actions, each with its own coverage area and path, different coverage areas and paths may correspond to different drying efficiencies. For example, normally it takes 10 minutes to bring the residual moisture on a newborn's body to a acceptable level. However, if a trainee fails to follow the prescribed path during training, resulting in a total wiping time of 20 minutes, the extended time may lead to hypothermia in the newborn. Therefore, to ensure the newborn's safety, the wiping time needs to be controlled. Since the total wiping time is a combination of the time consumed by multiple wiping actions, and because the absorption efficiency of the wiping material is limited, excessively fast wiping speed in a single action will increase the amount of residual moisture in the corresponding area. Therefore, the duration of each wiping action needs to be controlled. Given the need to control both the total wiping time and the duration of each wiping action, the coverage area and path of the wiping actions need to be rationally arranged to reduce the time wasted due to ineffective actions.

[0032] Therefore, the wiping actions in the teaching content are statistically analyzed in terms of path, coverage area, and duration along the corresponding path. This allows the system to understand the instructor's behavior in each area during the wiping process. For example, during head wiping, the wiping actions are statistically analyzed based on the instructor's wiping process. For instance, if the forehead is wiped first, then the top of the head, and finally the back of the head, then there are paths a, b, and c in this wiping process. When wiping the forehead, the wiping material stays on path a for 5 seconds. By combining path a and the stay time of 5 seconds, the nursing operation of wiping the forehead can be obtained. Similarly, this clarifies the instructor's dwell time and wiping path in each area of ​​the newborn throughout the entire drying process.

[0033] In step S3, colorimetric data is obtained from the teaching video data, and the colorimetric data is matched with the first teaching operation data to obtain the teaching standard, including the following steps: S311, based on the time sequence, the color development data is matched with the wiping action to determine the color development changes on the training carrier when the instructor performs the wiping action, and the color development change results are obtained; S312, if the change result of the color development data on the covered area corresponding to each wiping action in the first teaching operation data is a positive change, then the purpose of the wiping action is determined to be the wiping purpose; whereby, positive change refers to the color change that should occur in the reversible water-induced color-changing film when the moisture decreases.

[0034] S313, if the color development data of the covered area corresponding to the wiping action in the first teaching operation data is unchanged or reversed, then the purpose of the wiping action is determined to be a transitional purpose; wherein, reversed change refers to the color change that should occur in the reversible water-induced color-changing film when the moisture increases.

[0035] S314, based on the behavioral purpose of each wiping action and the corresponding color change results of each wiping action, the first teaching operation data is marked with corresponding wiping actions to obtain the teaching standard.

[0036] In this embodiment, by matching the color development data with the corresponding wiping action in time, the color data corresponding to the wiping action is determined, so that the judgment result is more accurate and effective when evaluating subsequent student training videos. At the same time, by utilizing the continuity of time, the color of the area covered by the wiping action is continuously judged to determine the color change of the covered area. Then, based on the relationship between the change of moisture and color on the reversible hydrochromic film, the behavioral purpose of each wiping action is determined, thereby determining the behavioral purpose of each wiping action in the teaching video. This further facilitates the system's understanding of the teaching video and improves the accuracy of the system's evaluation of student training videos based on the teaching video.

[0037] For example, after determining the event information of each wiping action during the wiping care instruction, it is necessary to further clarify the wiping purpose of each wiping action in the instructional video based on colorimetric data.

[0038] The primary goal of drying a newborn is to quickly remove moisture. However, each wipe targets a different area, and transitional actions occur between wiping sessions. Therefore, it's crucial to assess each wiping action to determine whether it's intended for drying or transitional. If it's for drying, the wiping action reduces moisture in the corresponding area, causing a change in color on the reversible hydrochromic film. For example, more moisture results in a darker color, while less moisture results in a lighter color.

[0039] Therefore, by combining the color development data in the teaching video data, the color development data is matched with the wiping action. When the wiping action is completed, if the color of the covered area becomes lighter, it is determined that the purpose of the corresponding wiping action is to dry the newborn's body. If the color of the covered area does not change, it is determined that the purpose of the corresponding wiping action is the transitional behavior of moving the wiping object from one area to another. This clarifies the behavioral purpose of each wiping action in the teaching video. After knowing the behavioral purpose and the wiping result corresponding to each wiping action, the first teaching operation data is marked with information to determine the final teaching standard, making the results evaluated based on the teaching standard more accurate.

[0040] In step S3, colorimetric data is obtained from the teaching video data, and the colorimetric data is matched with the first teaching operation data to obtain the teaching standard, including the following steps: S321, based on human dynamics, performs motion line judgment on wiping actions in teaching video data to determine the rationality of the wiping action's motion line; S322, If it is determined that the wiping action in the teaching video data does not have a reasonable movement path, then the wiping action is optimized according to the behavioral purpose of the wiping action to obtain the optimized behavior; S323, the optimized behavior is fed back to the instructor, who then conducts a teaching session based on the optimized behavior, resulting in the second optimized teaching result and the second optimized colorimetric data; S324, compare the color data of the second optimized color data with the color data of the first teaching operation data, compare the second optimized teaching results with the optimized behavior, and determine the optimization effect; S325, if the optimization effect is that the optimization behavior achieves better results, then the optimization behavior and the first optimized colorimetric data are selected as the teaching standard for subsequent teaching evaluation. S326. If the optimization effect is unsatisfactory, the original teaching video data will be used as the teaching standard for subsequent teaching evaluation. S327 If the optimization effect remains unchanged, obtain feedback from the instructors and select teaching behaviors based on the instructors' feedback to obtain the final teaching standard.

[0041] In this embodiment, by utilizing human dynamics, the wiping action and the continuity between actions in the teaching video are analyzed to determine the efficiency and effectiveness of the wiping action. When the wiping action in the teaching video is determined to lack a reasonable movement path, the unreasonable wiping action is optimized using human dynamics to obtain the optimized teaching behavior (optimized behavior). The optimized behavior is then fed back to the instructor, and the video data of the instructor implementing the optimized behavior is re-collected and compared with the original teaching video data to further verify the authenticity and effectiveness of the optimized behavior. This allows for optimal selection in subsequent teaching guidance and evaluation. At the same time, by obtaining feedback from the instructor when performing the optimized behavior, the rationality of the optimized behavior is further determined, thus ensuring that the optimized behavior is not only scientifically optimal but also perceptually reasonable.

[0042] For example, when instructors teach dry care techniques, there may be unnecessary or ineffective movements, or the instructors may have personal habits that lead to repetitive actions during the wiping process, resulting in reduced efficiency and increased error rates. Therefore, it is necessary to assess the rationality of the movement patterns in the instructor's teaching videos.

[0043] Since wiping a newborn is a dynamic physical act, the instructor's wiping movements in the instructional video can be analyzed using human dynamics to determine the continuity between consecutive wiping actions. For example, when wiping area A, if the instructor's limbs need to rotate their wrist 30 degrees to achieve the desired action, and assuming the normal wrist rotation angle is 20 degrees, then completing this action requires additional muscle control to achieve the 30-degree wrist rotation, thus determining that the action lacks a logical movement. Furthermore, assuming the hand is in a relaxed, slightly clenched state, while clenching it into a fist requires additional muscle control.

[0044] Meanwhile, when wiping the newborn, area a is wiped first, then area b, and finally area c. The path taken when wiping area b and then moving to area c is the same as the path taken when wiping area a and then moving to area b. Therefore, it can be determined that there is a problem with the movement path of the wiping action, that is, the movement path is unreasonable.

[0045] When the wiping action in the instructional video is deemed to lack logical movement, the corresponding wiping action is optimized to obtain a more reasonable wiping action. To ensure the authenticity and effectiveness of the optimized result, feedback is provided to the instructor, who then teaches the optimized action again. The results of the two lessons are compared to determine whether the optimized wiping action achieves a more efficient and accurate wiping effect. If the optimized action can complete wiping faster, is easier to master, or better absorbs and dries the water on the training medium, then the optimized action can be considered more effective. Therefore, the optimized action can be used as a teaching standard for instructional guidance and training evaluation to improve the training effect of trainees.

[0046] Furthermore, by utilizing feedback from instructors during the implementation of optimization behaviors, the rationality of these behaviors can be further determined, thereby ensuring that optimization behaviors not only guarantee scientific optimality but also ensure rationality in human perception.

[0047] In step S5, the substandard operation data is matched with the substandard color rendering data to determine the correspondence between the substandard operation data and the substandard color rendering data. Based on the correspondence, the student's correction data is determined, and an alarm signal is output based on the correction data. This includes the following steps: S51, if there are some substandard operation data that correspond one-to-one with substandard color development data, then corrective data is obtained based on the key events corresponding to the substandard operation data; S52, if some substandard operation data cannot be matched one-to-one with substandard color development data, then the non-matching substandard operation data is recorded as pairing failure operation data, and the non-matching substandard color development data is recorded as pairing failure color development data; wherein, pairing failure operation data refers to the case where the wiping action is substandard but the corresponding color development data is standard; pairing failure color development data refers to the case where the wiping action is standard but the corresponding color development data is substandard.

[0048] S53, if it is determined that there is pairing failure color display data, then based on the time data of the pairing failure color display data, determine whether there is pairing failure operation data in the time period before the time data of the pairing failure color display data. S54, if there is pairing failure operation data in the time period before the pairing failure color display data, then the key events corresponding to the pairing failure color display data and the pairing failure operation data are used as the student's correction data. S55, if there is no pairing failure operation data in the time period before the pairing failure color display data, then the key event corresponding to the pairing failure color display data is used as the student's correction data. S56 If it is determined that there is no pairing failure color display data but only pairing failure operation data, then the pairing failure operation data will not be marked as the student's correction data.

[0049] In this embodiment, the correspondence between substandard operation data and substandard color display data is determined by matching them. Substandard operation data with one-to-one correspondence is marked as correction data that needs to be corrected, thereby ensuring the training efficiency of trainees. Substandard operation data and substandard color display data that cannot be matched one-to-one are divided into categories, and the color display data that fails to be paired is judged and analyzed to determine the possible reasons for the formation of the failed pairing color display data. By matching the failed pairing operation data located before the time point of the corresponding failed pairing color display data, the correction data for the failed pairing color display data is determined, so as to comprehensively cover the reasons for substandard performance and improve the training effect of trainees.

[0050] For example, the following four situations may occur during the student's wiping process: 1. The student's wiping action and the color development result after wiping both meet the teaching standards; 2. The student's wiping action meets the teaching standards, but the color development result after wiping does not meet the teaching standards; 3. The student's wiping action does not meet the teaching standards, but the color development result after wiping meets the teaching standards; 4. Neither the student's wiping action nor the color development result meets the teaching standards.

[0051] In the first case, since all the teaching standards are met, the student's performance in the corresponding wiping action can be judged as qualified. In the second scenario, there may be reasons such as insufficient wiping force, ineffective wiping, or poor water absorption of the wiping material, which may result in the wiping action being correct but the wiping result not meeting the standard. Therefore, it is necessary to analyze and judge to determine the specific reasons and point them out to correct the trainee's behavior.

[0052] In the third case, since the judgment of newborn drying care is result-oriented, if the color development result meets the teaching standard, it means that the student's final drying behavior also meets the drying requirements, so the student's wiping action can be judged as qualified.

[0053] In the fourth case, since both the wiping action and the color development result do not meet the teaching standards, it can be determined that the reason for the unqualified color development result is that the wiping action is unqualified. Therefore, the wiping action can be corrected first so that the trainees can perform the wiping action correctly.

[0054] The relationships between the substandard operation data and the substandard color development data for the four scenarios mentioned above are as follows: For the first scenario: there are no substandard operation data or substandard color development data; For the second scenario: at least one substandard operation data and one substandard color development data do not correspond; For the third scenario: at least one substandard color development data and one substandard operation data do not correspond; For the fourth scenario: there is a one-to-one correspondence between the substandard operation data and the substandard color development data.

[0055] Therefore, once it is determined that there are substandard operation data and substandard color development data, the relationship between the substandard operation data and the substandard color development data is determined by comparing them one by one. If the substandard operation data and the substandard color development data correspond one by one, it indicates that the student's operation problem belongs to the fourth situation. Therefore, it is only necessary to give the student a prompt based on the substandard operation data.

[0056] If there is no one-to-one correspondence between substandard operation data and substandard color development data, further judgment is required to determine the possible causes of this correspondence, so that targeted corrections can be made based on the causes.

[0057] By utilizing a one-to-one correspondence, substandard operational data and substandard color development data are categorized into failed pairing operational data and failed pairing color development data. When failed pairing operational data exists, since the corresponding color development data meets the standard, the behavior can be determined to be a personal behavioral problem of the trainee. When failed pairing color development data exists, it may be due to the ineffective execution of the corresponding wiping action, or it may be caused by other wiping actions at a time point prior to this wiping action being substandard. These other wiping actions at a time point prior to this wiping action include wiping actions with a corresponding relationship and wiping actions without a corresponding relationship. Wiping actions with a corresponding relationship need to be corrected because their color development data is substandard. At the same time, behaviors without wiping actions also need to be corrected, thereby ensuring a comprehensive reduction in the impact of substandard wiping actions and improving the accuracy of evaluating and judging trainee training results.

[0058] In step S324, the colorimetric data of the second optimized colorimetric data are compared with the colorimetric data of the first teaching operation data, and the second optimized teaching results are compared with the optimized behavior to determine the optimization effect, including the following steps: S324a, compare the second optimized teaching result with the optimized behavior by wiping action, determine the similarity between the wiping action of the instructor and the wiping action in the optimized behavior, and obtain the execution perfection data; S324b, if the perfection data of the instructor's execution of the optimization behavior is greater than the built-in standard threshold, then the color depth of the second optimization color data is compared with the color data of the first teaching operation data. S324c, If the color depth of the second optimized color data is lighter than the color data in the first teaching operation data, then the optimized behavior is determined to have a better wiping effect; S324d, if the color depth of the second optimized color data is deeper than the color data in the first teaching operation data, then the wiping effect of the optimization behavior is determined to be poor; S324e, if the color depth of the second optimized color data is the same as the color data in the first teaching operation data, then it is determined that the wiping effect of the optimized behavior is the same. S324f, obtain the execution time of the optimization behavior, and compare the execution time of the optimization behavior with the execution time in the first teaching operation data; S324g, if the execution time of the optimization behavior is longer, the optimization behavior is judged to be ineffective; S324h, if the execution time of the optimized behavior is equal to the execution time in the first teaching operation data, then the execution effect of the optimized behavior is determined to be unchanged; S324i, if the execution time of the optimization behavior is shorter, then the execution effect of the optimization behavior is determined to be better; S324j, when the wiping effect of the optimization behavior is better, and the execution effect is either better or unchanged, the optimization effect is determined to be better; S324k: When the wiping effect of the optimized behavior is better, but the execution effect is poor, an alarm signal for re-collection will be output. S324l, when the wiping effect of the optimized behavior is the same, and the execution effect is better, then the optimization effect is determined to be better; S324m: When the wiping effect of the optimized behavior is the same, but the execution effect is poor, the optimization effect is judged to be poor. S324n, when the wiping effect of the optimization behavior is the same, but the execution effect is unchanged, the optimization effect is determined to be unchanged; S324o, when the wiping effect of the optimized behavior is poor, but the execution effect is better, then output the alarm signal to be re-acquired; S324p states that if the wiping effect of the optimization behavior is poor, and the execution effect is poor or unchanged, then the optimization effect is determined to be poor.

[0059] For example, when evaluating the optimized behavior, the instructor's wiping action is first compared with the optimized behavior to ensure that the instructor perfectly follows the optimized behavior in drying, thus guaranteeing the effectiveness of the second optimized teaching result. After determining that the wiping operation performed by the instructor according to the optimized behavior is effective, the optimized operation result is then evaluated to ensure the effectiveness of the evaluation result. The drying effect is determined by comparing the second optimized teaching result with the first teaching operation data through colorimetric analysis, and the execution time is then compared to determine the execution efficiency. When the drying result brought by the optimized behavior is both fast wiping and fast drying, it indicates that the optimized behavior has achieved better results. Conversely, when the drying result is slow wiping and poor dryness, it indicates that the optimized behavior has achieved poor results. Therefore, the optimization effect of the optimized behavior can be comprehensively judged, improving the accuracy of the comprehensive evaluation of the optimized behavior.

[0060] The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment also includes the following steps: S71, obtain the student's student information, and read the corresponding student's training video data and its corresponding correction data based on the student's student information; S72, Perform mathematical statistics on the correction data of the trainees to determine the high-frequency correction data and record it as the correction data of the trainees to be analyzed; wherein, the high-frequency correction data refers to the correction data that appears more frequently than the average frequency among all the correction data of the trainees to be analyzed.

[0061] S73, Based on the correction data of the student to be analyzed, determine the corresponding training video data and record it as the training data of the student to be analyzed; S74, perform wiping motion learning on the training data of the student to be analyzed to obtain a wiping motion model for the student to be analyzed; where the wiping motion model refers to the maximum state that the student can achieve when performing the corresponding wiping motion. For example, the degree of wrist flexion: if the wrist can normally bend to 10 degrees, but the student's maximum flexion is only 5 degrees, then the wiping motion model is a wrist flexion of 0~5 degrees. The variation within 0~5 degrees conforms to the student's normal state, while when it exceeds 5 degrees, it does not conform to the student's normal state.

[0062] S75, compare the wiping action model of the trainee to be analyzed with the corresponding wiping action in the teaching standard to determine the differences; S76, based on human dynamics and distinguishing points, simulates the wiping action model, determines that while keeping the wiping action model unchanged, the limb movements are adjusted to make the wiping action model meet the wiping action requirements in the teaching standard, and obtains limb guidance data; S77, obtain the time point corresponding to the student's correction data to be analyzed, and obtain the wiping action before the corresponding time point to obtain the wiping action to be adjusted; S78 adjusts the wiping action to be adjusted based on limb guidance data to obtain a new teaching standard, and guides and evaluates the training of students based on the new teaching standard.

[0063] In this embodiment, the training status of each student is determined by acquiring their information. When a student makes frequent errors, a targeted analysis is conducted based on these errors, and the causes of the problems are addressed to ensure the effectiveness of subsequent training. A machine learning algorithm is used to learn from the training data of the students being analyzed, thereby determining a wiping action model that conforms to the student's wiping habits. By comparing this wiping action model with the corresponding wiping action in the teaching standard, the differences between the two are identified, clarifying the correction targets. Then, combined with human dynamics, the student's wiping action model is simulated based on these differences. This ensures that while maintaining the wiping action model, adjustments to limb movements are made to achieve the specified requirements, thus guaranteeing the accuracy of the student's wiping actions, reducing humidity, protecting training results, and improving training effectiveness.

[0064] For example, different trainees may have deficiencies in certain wiping movements due to their behavioral habits, making it impossible to meet the wiping requirements in the teaching standards. For instance, if the instructor is right-handed and the trainee is left-handed, the trainee may deviate from certain movements when practicing wiping care based on a right-handed instructional video because their right hand is not their dominant hand. In this case, it is necessary to guide the trainee's movements while taking into account their left-handedness to ensure that they can meet the wiping care requirements.

[0065] First, based on the trainee's information, it's determined whether the trainee frequently makes the same error during training. If so, it indicates an inherent error in the trainee's wiping action, such as control deviation due to the non-dominant hand. After identifying the inherent error in a particular wiping action, machine learning is used to construct a realistic motion model of the trainee performing the action. This model allows for the determination of the trainee's control ability, such as finger dexterity and the degree of hand flexion. After determining the trainee's control ability, the realistic motion model is compared with the wiping action in the teaching standard to identify discrepancies, which are areas requiring further adjustment. Finally, using human dynamics, while ensuring the trainee's range of motion and control ability remain within their controllable range, multi-directional adjustments to the limbs allow the trainee to compensate for these discrepancies, ensuring they meet the teaching standard requirements when performing the wiping action. For example, if the wrist rotation angle is insufficient for the wiping action, the angle of the arm and torso can be adjusted to ensure that the wrist rotation angle remains constant while the wrist reaches a specific angle. This allows for targeted adjustments to the teaching standards based on the student's situation, resulting in optimal training and evaluation.

[0066] Compared with existing neonatal care assessment methods based on image and dry / wet color fusion, this invention improves the accuracy of training assessment results.

[0067] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A neonatal care assessment method based on image and dry / wet colorimetric fusion judgment, characterized in that, include: Based on a newborn simulation body and a reversible water-induced color-changing film, a drying training carrier was constructed. Videos of the process of instructors using the drying training carrier to teach drying were collected to obtain teaching video data. Obtain behavioral tagging data of the instructor on the teaching video data, perform action analysis on the teaching video data based on the behavioral tagging data, determine the key events in the teaching video data and the key information of the key events, and obtain the first teaching operation data; Obtain colorimetric data from the teaching video data, and match the colorimetric data with the first teaching operation data to obtain the teaching standard; Acquire students' training video data and perform real-time operation matching and color matching with teaching standards to obtain substandard operation data and substandard color data; The substandard operation data is matched with the substandard color rendering data to determine the correspondence between the substandard operation data and the substandard color rendering data. Based on the correspondence, the corrective data of the trainee is determined, and an alarm signal is output based on the corrective data. Obtain student information and determine the student's historical correction data during the training process based on the student information. Perform a time-series judgment between the historical correction data and the student's current operation. If there is a time-series adjacency between the current operation and the historical correction data, provide an early warning to the student based on the historical correction data after the student completes the current operation.

2. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 1, characterized in that: The process of acquiring behavioral marker data of the instructor on the teaching video data, performing action analysis on the teaching video data based on the behavioral marker data, determining key events and key information of the key events in the teaching video data, and obtaining first teaching operation data includes: The teaching video data is split based on the behavioral tagging data to obtain the teaching content under the corresponding tags; Based on the time sequence of the teaching video data in the teaching content, the operation timing data is obtained; The wiping actions under each marked teaching content are statistically analyzed in terms of path and coverage to obtain wiping action data; The duration of the wiping action is statistically analyzed to determine the duration of the wiping action. The data on wiping actions, duration, and sequence of operations are recorded as the first teaching operation data for neonatal drying care.

3. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 1, characterized in that: The step of acquiring colorimetric data from teaching video data and matching the colorimetric data with first teaching operation data to obtain teaching standards includes: Based on the time sequence, the color development data is matched with the wiping action to determine the color development changes on the training carrier when the instructor wipes it, and the color development change results are obtained. If the color development data of the covered area corresponding to each wiping action in the first teaching operation data changes in a positive direction, then the purpose of the wiping action is determined to be the wiping purpose. If the colorimetric data of the area covered by the wiping action in the first teaching operation data remains unchanged or changes in the opposite direction, then the purpose of the wiping action is determined to be a transitional purpose. Based on the behavioral purpose of each wiping action and the corresponding color change results of each wiping action, the first teaching operation data is marked with corresponding wiping actions to obtain the teaching standard.

4. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 3, characterized in that: The step of acquiring colorimetric data from teaching video data and matching the colorimetric data with first teaching operation data to obtain teaching standards further includes: Based on human dynamics, the movement lines of wiping actions in teaching video data are judged to determine the rationality of the wiping action movement lines; If it is determined that the wiping action in the teaching video data does not have a reasonable movement path, then the wiping action is optimized according to the behavioral purpose of the wiping action to obtain the optimized behavior; The optimized behavior is fed back to the instructor, who then conducts a lesson based on the optimized behavior, resulting in a second optimized teaching result and second optimized colorimetric data. The second optimized colorimetric data is compared with the colorimetric data of the first teaching operation data, and the second optimized teaching results are compared with the optimized behaviors to determine the optimization effect; If the optimization effect is that the optimization behavior achieves better results, then the optimization behavior and the first optimized color data are selected as the teaching standard for subsequent teaching evaluation. If the optimization effect is unsatisfactory, the original teaching video data will be used as the teaching standard for subsequent teaching evaluation. If the optimization effect remains unchanged, then obtain feedback from the instructors and select teaching behaviors based on their feedback to obtain the final teaching standard.

5. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 1, characterized in that: The process of matching substandard operation data with substandard color rendering data to determine the correspondence between them, determining the student's correction data based on the correspondence, and outputting an alarm signal based on the correction data includes: If there is a one-to-one correspondence between some substandard operation data and substandard color development data, then corrective data is obtained based on the key events corresponding to the substandard operation data; If some substandard operation data cannot be matched one-to-one with substandard color development data, then the non-matching substandard operation data will be recorded as pairing failure operation data, and the non-matching substandard color development data will be recorded as pairing failure color development data. If it is determined that there is pairing failure color display data, then based on the time data of the pairing failure color display data, it is determined whether there is pairing failure operation data in the time period before the time data of the pairing failure color display data. If there is pairing failure operation data in the time period before the pairing failure color data, the key events corresponding to the pairing failure color data and the pairing failure operation data will be used as the student's correction data. If there is no pairing failure operation data in the time period before the pairing failure color data, the key event corresponding to the pairing failure color data will be used as the student's correction data. If it is determined that there is no pairing failure display data but only pairing failure operation data, then the pairing failure operation data will not be marked as the student's correction data.

6. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 4, characterized in that: The step of comparing the color development data of the second optimized colorimetric data with the colorimetric data of the first teaching operation data, and comparing the second optimized teaching results with the optimized behavior to determine the optimization effect includes: The second optimized teaching result is compared with the optimized behavior by wiping action to determine the similarity between the wiping action of the instructor and the wiping action in the optimized behavior, and the execution perfection data is obtained. If the instructor's execution perfection data for the optimization behavior is greater than the built-in standard threshold, the color depth of the second optimization color data is compared with the color data of the first teaching operation data. If the color depth of the second optimized color data is lighter than the color data in the first teaching operation data, then the optimized behavior is judged to have a better wiping effect; If the color depth of the second optimized color data is deeper than that of the color data in the first teaching operation data, then the wiping effect of the optimization behavior is determined to be poor. If the color depth of the second optimized color data is the same as the color data in the first teaching operation data, then the wiping effect of the optimized behavior is determined to be the same. Obtain the execution time of the optimized behavior and compare it with the execution time in the first teaching operation data; If the execution time of the optimization behavior is longer, the optimization behavior is judged to be ineffective. If the execution time of the optimized behavior is equal to the execution time in the first teaching operation data, then the execution effect of the optimized behavior is determined to be unchanged; If the execution time of the optimization behavior is shorter, then the optimization behavior is considered to be more effective. When the wiping effect of the optimization behavior is better, and the execution effect is either better or unchanged, the optimization effect is determined to be better. When the wiping effect of the optimization behavior is better, but the execution effect is poor, an alarm signal for re-collection will be output. When the wiping effect of the optimization behavior is the same, and the execution effect is better, then the optimization effect is determined to be better; When the wiping effect of the optimization behavior is the same, but the execution effect is poor, the optimization effect is judged to be poor. When the wiping effect of the optimization behavior is the same, but the execution effect is unchanged, the optimization effect is determined to be unchanged. When the wiping effect of the optimization behavior is poor, but the execution effect is better, an alarm signal for re-collection will be output. When the wiping effect of the optimization behavior is poor, and the execution effect is poor or unchanged, the optimization effect is judged to be poor.

7. The neonatal care assessment method based on image and dry / wet colorimetric fusion judgment according to claim 1, characterized in that: Also includes: Obtain the student's information, and based on the student's information, read the corresponding student's training video data and its corresponding correction data; Statistical analysis was performed on the correction data of the trainees to identify the frequently occurring correction data, which was then recorded as the correction data of the trainees to be analyzed. Based on the correction data of the students to be analyzed, determine the corresponding training video data and record it as the training data of the students to be analyzed. The wiping motion model of the student to be analyzed is obtained by learning the wiping motion from the student's training data. The wiping action model of the trainee to be analyzed is compared with the corresponding wiping action in the teaching standard to identify the differences; Based on human dynamics and distinguishing points, the wiping action model was simulated to determine how to make the wiping action model meet the wiping action requirements in the teaching standard by adjusting the limb movements while keeping the wiping action model unchanged, and thus obtain limb guidance data. Obtain the time point corresponding to the student's correction data to be analyzed, and obtain the wiping action before the corresponding time point to obtain the wiping action to be adjusted; Based on the body guidance data, the wiping action to be adjusted is adjusted to obtain a new teaching standard, and the training guidance and evaluation of students are based on the new teaching standard.

Images