Experimental step automatic scoring and optimizing method based on image recognition

Abstract

The invention discloses an automatic scoring and optimizing method for experimental steps based on image recognition, which comprises the following steps of: dividing a plurality of steps of an experimental process into a front step set and a sequential step set according to a sequential relationship, extracting characteristic actions of experimenters, and respectively comparing the characteristic actions with specified actions in the front steps and the sequential steps, so as to improve the scoring accuracy of the experimenters. Scoring is performed according to whether the characteristic action accords with the specified action or not to ensure that the operation of an experimenter accords with the specification, and when the characteristic action accords with the specified action, further dynamic scoring is performed according to the time range and the sequence relation of the action, so that the recognition capability of complex experiment operation can be improved through staged judgment; according to the method, the accuracy of scoring is guaranteed, the fineness and the accuracy of scoring are guaranteed, a plurality of steps can be accurately monitored, omission or misoperation is prevented, and the fault tolerance and the adaptability of scoring are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of experimental scoring methods, and in particular to an experimental step automatic scoring and optimization method based on image recognition. BACKGROUND

[0002] In the fields of chemical, biological and other experimental sciences, the operation specification and step execution quality of experimenters directly affect the results and effectiveness of experiments. In current chemical tests or experiments, manual observation and evaluation of students' operations are often required to form the final experimental scores.

[0003] However, this traditional experimental scoring method requires experimenters or instructors to score the experimental operation process based on their own experience and judgment. However, manual scoring is easily influenced by subjective bias, and in complex experimental processes, manual scoring may not be able to fully capture all operational details, making it difficult to ensure the fairness and accuracy of the scoring.

[0004] At the same time, the time requirements of different steps may differ in the experimental operation process. Some steps require completion within a specific time range, while other steps may not have strict time requirements. It is often difficult to accurately grasp the operation time during manual scoring, resulting in scoring errors. In particular, when there are many experimental steps, the time tolerance ranges are different, and manual scoring cannot accurately distinguish the time differences of each step, resulting in inaccurate scoring. Therefore, it is necessary to design an experimental step automatic scoring and optimization method based on image recognition to solve the above problems. SUMMARY

[0005] The present application overcomes the shortcomings of the prior art and provides an experimental step automatic scoring and optimization method based on image recognition.

[0006] To achieve the above purpose, the technical solution adopted by the present application is as follows: an experimental step automatic scoring and optimization method based on image recognition, comprising the following steps:

[0007] Step S1, decompose the experimental process into a set of pre-step and a set of sequential steps with time sequence correlation, each step containing a specified action with spatiotemporal constraint parameters;

[0008] Step S2, identify several characteristic actions of the experimenter, calculate the matching degree of each characteristic action with the specified action, and determine it as a compliant action if it meets the threshold condition, and mark it as a pre-score item;

[0009] Step S3, determine whether the compliant action belongs to the pre-step set or the sequential step set, obtain the action time of the compliant action, and dynamically calculate the score loss amount according to the judgment result and the spatiotemporal constraint parameters of the corresponding specified action of the compliant action;

[0010] Step S4: Based on the judgment result, calculate the completeness of the set of preceding steps or the set of sequential steps, output the unrecognized prescribed actions, and score them;

[0011] Step S5: Combine the scores from steps S3 and S4 to obtain the final score and step completeness.

[0012] In a preferred embodiment of the present invention, in step S1, based on the temporal correlation of the experimental steps, several steps of the experimental process are divided into a set of prerequisite steps and a set of sequential steps. The prerequisite steps are the prerequisites for the sequential steps, and there are several parallel operation steps in the prerequisite steps.

[0013] Each step includes a prescribed action with spatiotemporal constraints. Specifically, the preceding steps and sequential steps are divided into several characteristic actions, which are used as prescribed actions. The start and end anchor points of each characteristic action are defined, and the time points of the start and end anchor points are recorded as the time range of the characteristic action.

[0014] In a preferred embodiment of the present invention, in step S2, both the characteristic action and the prescribed action include: the movement trajectory of the equipment.

[0015] In a preferred embodiment of the present invention, step S2 includes the following sub-steps:

[0016] Step S21: Establish a unified spatiotemporal coordinate system and obtain the hand movements of the equipment movement trajectory in the characteristic movements, including the positions of key points of the wrist, fingertips, and elbow.

[0017] Step S22: Record the time points of adjacent key points;

[0018] Step S23: Based on real-time tracking and recording of key point position changes, and recording the positions of all key points as feature actions, using formula A... i (T)={(x1,y1,z1), (x2,y2,z2),…, (x i ,y i ,z i The expression is as follows:} where A i (T) represents the position of all keypoints at time t, (x) represents the position of all keypoints at time t. i ,y i ,z i () indicates the position of a key point at time t;

[0019] Step S24: Compare the feature action with the preset prescribed action to determine whether the feature action belongs to the prescribed action.

[0020] In a preferred embodiment of the present invention, in step S21, identifying several characteristic actions of the experimenter includes the following steps:

[0021] Step S211, obtaining several two-dimensional images of the personnel action and the equipment state;

[0022] Step S212, converting the several two-dimensional images into three-dimensional space coordinates, and selecting the center of the experimental table as the coordinate system origin;

[0023] Step S213, mapping the personnel action and the equipment position into the coordinate system, and recording the position of the equipment in the image data in real time.

[0024] In a preferred embodiment of the present application, in the step S24, the position A' of the key point in the t moment in the defined specified action is included i (T) = {(x'1, y'1, z'1), (x'2, y'2, z'2),..., (x'n, y'n, z'n)}, the direct distance between the actual key point and the specified action in the t moment is calculated t , t , t . Where (x, y, z) and (x', y', z') are the space coordinates of the characteristic action and the specified action at the i moment, respectively; i , i , i , t , t , t .

[0025] d is the Euclidean distance, and a threshold value is set. If d is less than or equal to the preset threshold value, it is considered that the characteristic action matches the specified action and meets the specified action. If d is greater than the preset threshold value, it is considered that the characteristic action does not meet the specified action.

[0026] In a preferred embodiment of the present application, in the step S3, if d is less than or equal to the preset threshold value, it is considered that the characteristic action matches the specified action and meets the specified action, and it is judged whether the step to which the characteristic action belongs is a pre-step set. If there is a pre-step set, the time of the characteristic action is further compared with the time of the specified action;

[0027] The time of the starting anchor point of each characteristic action is defined as Ti, and the time of the ending anchor point is defined as Ty. The time of the specified action is ΔT = ||Ty-Ti||. The starting time and the ending time of the characteristic action are recorded, the actual time Δt is calculated, and the actual time Δt is compared with the time of the specified action. If the actual time is within the specified time range, the characteristic action is recorded as a score;

[0028] If the characteristic action belongs to the prescribed action and the action characteristic belongs to the sequential step, it is determined whether the preceding step of the sequential step is completed, if completed and the time of the characteristic action in the sequential step is less than the time of the prescribed action, it is summarized in the score table, and the score loss amount is dynamically calculated according to the judgment result and the time and space constraint parameters of the prescribed action corresponding to the compliance action, if not completed, it is determined that the sequential step cannot score.

[0029] In a preferred embodiment of the present application, in the step S3, the maximum tolerance time Δt max is defined. max If the characteristic action belongs to the prescribed action, the actual time Δt of the characteristic action exceeds the time ΔT of the prescribed action, and the actual time Δt of the characteristic action is less than or equal to the maximum tolerance time Δt

[0030] In a preferred embodiment of the present application, in the step S3, the time error is calculated based on the actual time Δt and the time ΔT of the prescribed action, and the calculation of the score loss amount includes,

[0031] ΔT-Δt is the time error.

[0032] If the time error is within the maximum tolerance time, the deduction ratio is Δt max -ΔT, the score is 1-deduction ratio, and if the time error exceeds the maximum tolerance time, the score is 0.

[0033] In a preferred embodiment of the present application, in the step S3, if d is greater than a preset threshold value, it is considered that the characteristic action does not match the prescribed action, the characteristic action does not conform to the prescribed action, and it is determined that the characteristic action cannot score.

[0034] The present application solves the defects in the background art, and has the following beneficial effects:

[0035] (1) The present application provides an experimental step automatic scoring and optimization method based on image recognition, which divides a number of steps of an experimental process into a preceding step set and a sequential step set according to the sequential relationship, extracts the characteristic action of an experimental personnel, compares it with the prescribed action in the preceding step and the sequential step respectively, scores according to whether the characteristic action conforms to the prescribed action, ensures that the operation of the experimental personnel conforms to the prescription, when the characteristic action conforms to the prescribed action, further dynamically scores according to the time range and the sequential relationship of the action, judges by stages, not only improves the recognition ability of complex experimental operation, but also ensures the fineness and accuracy of the score, at the same time, can accurately monitor a number of steps, prevent omissions or misoperations, and improve the fault tolerance and adaptability of the score.

[0036] (2) The application provides an image recognition-based automatic scoring and optimization method for experimental steps, which can accurately capture the details of each action by tracking the movement trajectories of the hands, limbs and equipment of the experimental personnel in real time, thereby guaranteeing the accuracy and integrity of the experimental operation, and dynamically comparing the pre-set specified action, quantitatively scoring each characteristic action of the experimental steps, and optimizing according to the quality, time and multi-dimensional factors of the action, thereby avoiding the deviation or subjective judgment that may occur in manual scoring, and providing a standardized scoring method.

[0037] (3) The application provides an image recognition-based automatic scoring and optimization method for experimental steps, which can accurately capture the details of each action by tracking the movement trajectories of the hands, limbs and equipment of the experimental personnel in real time, thereby guaranteeing the accuracy and integrity of the experimental operation, and dynamically comparing the pre-set specified action, quantitatively scoring each characteristic action of the experimental steps, and optimizing according to the quality, time and multi-dimensional factors of the action, thereby avoiding the deviation or subjective judgment that may occur in manual scoring, and providing a standardized scoring method. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief introductions will be given below to the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments described in the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings;

[0039] Figure 1 is a flowchart of the image recognition-based automatic scoring and optimization method for experimental steps of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0042] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the scope of protection of the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified and limited, the term "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0043] In the description of the present application, it should be noted that unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0044] As shown in Figure 1 The image recognition-based experimental step automatic scoring and optimization method includes the following steps:

[0045] Step S1, decompose the experimental procedure into a set of pre-step and a set of sequential steps, each step contains a specified action with space-time constraint parameters; the experiment is divided into several steps, according to the order relationship between the steps, the several steps of the experimental procedure are divided into a set of pre-steps and a set of sequential steps, and each step is divided into several characteristic actions, each characteristic action corresponds to a corresponding time range;

[0046] Step S2, identify several characteristic actions of the experimental personnel, calculate the matching degree of each characteristic action and the specified action, and determine the compliance action if the threshold condition is met, and mark it as a pre-score item;

[0047] Step S3, determine whether the compliance action belongs to the pre-step set or the sequential step set, obtain the action time of the compliance action, and dynamically calculate the score loss according to the judgment result and the space-time constraint parameters of the specified action corresponding to the compliance action;

[0048] Step S4, calculate the completeness of the pre-step set or the sequential step set according to the judgment result, and output the un-identified specified action and score.

[0049] Step S5, the scores of step S3 and step S4 are integrated to obtain the final score and step completeness.

[0050] In the step S1, based on the sequence relationship between the experimental steps, the experimental steps are divided into a pre-step set and a sequence step set;

[0051] The pre-step is the precondition of the sequence step, that is, the operation that must be completed before the sequence step, and there are several parallel operation steps in the pre-step, so the pre-step is generally a step with relatively simple operation, or a step in which multiple operations need to be performed in parallel. After the pre-step is completed, the sequence step can start. For example, in a chemical experiment, the preparation and preliminary processing of substances such as "sampling", "cleaning equipment", and "preparing solution" are pre-steps, which are necessary operations before the sequence steps such as "heating solution", "mixing reagent", and "measuring reaction temperature".

[0052] And the pre-step in the experiment involves multiple operations, which not only need to be performed before the sequence step, but also may have the need for parallel operation, which means that certain pre-steps can be performed simultaneously to save time and improve experimental efficiency, such as:

[0053] Pre-step 1: Prepare reagent A (in progress);

[0054] Pre-step 2: Prepare reagent B (parallel);

[0055] Pre-step 3: Clean the equipment (in progress).

[0056] Some steps in the pre-step set can be performed in parallel, and finally all pre-steps are completed before the sequence step is executed. The parallelism of the pre-step is particularly important in some complex experiments, which improves the experimental efficiency and reduces the invalid waiting time. The flexibility and diversity in step scoring improve the adaptability of the scoring mechanism.

[0057] Among them, the relationship between the pre-step and the sequence step is not a simple linear relationship, and the pre-step can also be the sequence step of other steps, that is, in the pre-step set of the sequence step in the same experiment, the pre-step is the precondition of other steps and is also the sequence step of other steps, such as:

[0058] Experimental step 1: Add reagent A to the reaction bottle;

[0059] Experimental step 2: Heat the reaction bottle;

[0060] Experimental step 3: Add catalyst.

[0061] In this context, "adding a catalyst" can be a prerequisite for "heating the reaction flask" (i.e., the reaction flask must be heated before the catalyst can be added), but "adding a catalyst" itself is also a sequential step in the subsequent "observing the reaction." In other words, "adding a catalyst" can be both a prerequisite for one step and a sequential step for another. This cross-relationship ensures the flexibility of the scoring mechanism and encourages the scoring system to comprehensively consider the completion of multiple prerequisite steps when considering each experimental step, thereby more comprehensively judging the rationality of the sequential steps and the performance of the experimenters.

[0062] The pre-step and sequential steps are divided into several feature actions. The start anchor point and end anchor point of each feature action are defined, and the time points of the start anchor point and end anchor point are recorded as the time range of the feature action.

[0063] After breaking down the experimental steps, the next step is to further break down each step into characteristic actions. Each characteristic action represents a smaller unit action in the experimental operation. This characteristic action includes the trajectory of equipment movement and the hand movements of the experimenter. Each characteristic action corresponds to a specific time range, namely the start anchor point and the end anchor point of the characteristic action. For example, if the experimental step is "pouring liquid", then the characteristic actions include: Characteristic action 1: picking up the container (time range: T1-T2); Characteristic action 2: pouring the liquid into another container (time range: T3-T4); Characteristic action 3: putting down the container (time range: T5-T6). This ensures that the time nodes of each action are accurately tracked, and then the actions are scored.

[0064] In step S2 of this invention, both the characteristic action and the prescribed action include: the movement trajectory of the equipment;

[0065] Step S2 includes the following sub-steps:

[0066] Step S21: Establish a unified spatiotemporal coordinate system and obtain the hand movements of the equipment movement trajectory in the characteristic movements, including the positions of key points of the wrist, fingertips, and elbow.

[0067] Step S22: Record the time points of adjacent key points;

[0068] Step S23: Based on real-time tracking and recording of key point position changes, and recording the positions of all key points as feature actions, using formula A... i (T)={(x1,y1,z1), (x2,y2,z2),…, (x i ,y i ,z i The expression is as follows:} where A i (T) represents the position of all keypoints at time t, (x) represents the position of all keypoints at time t. i ,yi ,z i () indicates the position of a key point at time t;

[0069] Step S24: Compare the feature action with the preset prescribed action to determine whether the feature action belongs to the prescribed action.

[0070] In step S21 of this invention, a unified spatiotemporal coordinate system is established, including the following steps:

[0071] Step S211: Acquire several two-dimensional images of personnel movements and equipment status;

[0072] Two-dimensional images of the experimenters and their equipment are acquired in real time using image acquisition devices. These two-dimensional images include the actions of the experimenters and the specific positions of the equipment, as well as their relative changes during the experiment.

[0073] Step S212: Convert several two-dimensional images into three-dimensional spatial coordinates, and select the center of the experimental platform as the origin of the coordinate system;

[0074] Two-dimensional images provide a preliminary data source for subsequent action recognition and analysis. In order to more accurately describe the spatial distribution of actions and the position of equipment, the data obtained from the two-dimensional images will be mapped to three-dimensional space and the two-dimensional image data will be converted into three-dimensional coordinates, so as to accurately locate and track the position of actions and equipment. In this coordinate system, the center of the experimental platform is selected as the origin of the coordinate system.

[0075] Step S213: Map personnel movements and equipment positions to a coordinate system and record the position of equipment in the image data in real time.

[0076] Step S24 includes defining the position A' of the key point in the specified action at time t. i (T)={(x'1,y'1,z'1),(x'2,y'2,z'2),…,(x' t ,y' t ,z' t )}, calculate the direct distance between the actual key point and the prescribed action at time t. Where (x) i ,y i ,z i ) and (x' t ,y' t ,z' t (i) represents the spatial coordinates of the characteristic action and the prescribed action at time i.

[0077] d is the Euclidean distance, and a threshold value is set. If d is less than or equal to the preset threshold value, it is considered that the characteristic action matches the prescribed action and meets the prescribed action. If d is greater than the preset threshold value, it is considered that the characteristic action does not meet the prescribed action.

[0078] By accurately extracting the characteristic action of the experimental personnel, all key operation steps in the experimental process are comprehensively identified. The accurate identification of the action is the basis for subsequent scoring, which can avoid the errors or subjective bias easily occurring in the traditional manual scoring method. The acquired characteristic steps are compared with the preset prescribed action, and whether the characteristic action meets the prescribed action is determined according to the matching degree of each action. Then, the characteristic action is scored to ensure the consistency and fairness of the scoring. Moreover, the prescribed action is used as the standard, which can eliminate the scoring difference between different experimental personnel, improve the objectivity and consistency of the scoring, and avoid the simple scoring method by whether the step is completed, thereby improving the accuracy of the scoring.

[0079] It should be noted that during the experiment, whether the characteristic action is the prescribed action is recorded in real time. If the characteristic action of the experimental personnel deviates from the predetermined trajectory, it is recorded, which provides a basis for the experimental personnel to find operation errors subsequently. Then, the experimental personnel can find and correct the operation errors.

[0080] In the step S3, if d is less than or equal to the preset threshold value, it is considered that the characteristic action matches the prescribed action and meets the prescribed action. Whether the step to which the characteristic action belongs is a set of preconditions is determined. If there is a set of preconditions, the time of the characteristic action is further compared with the time of the prescribed action.

[0081] The time of the starting anchor point of each characteristic action is defined as Ti, and the time of the ending anchor point is defined as Ty. The time of the prescribed action is ΔT = ||Ty-Ti||. The starting time and the ending time of the characteristic action are recorded. The actual time Δt is calculated. The actual time Δt is compared with the time of the prescribed action. If the actual time is within the prescribed time range, the characteristic action is scored.

[0082] If the characteristic action belongs to the prescribed action and the action characteristic belongs to the sequential step, whether the precondition of the sequential step is completed is determined. If it is completed and the time of the characteristic action in the sequential step is less than the time of the prescribed action, it is summarized in the scoring table. According to the judgment result and the space-time constraint parameter of the prescribed action corresponding to the compliant action, the score loss is dynamically calculated. If it is not completed, the sequential step is not scored.

[0083] In step S3, the characteristic action is judged according to the judgment standard of step S2, if the characteristic action is judged to be in conformity with the prescribed action, it is further judged whether the step to which the characteristic action belongs is a preceding step, if it is a preceding step, the time of the characteristic action is compared with the time of the prescribed action, the actual time of the characteristic action must be less than or equal to the time of the prescribed action, otherwise it will not conform to the prescribed action;

[0084] By accurately controlling the time range, the operation of the experimental personnel can be accurately evaluated, and the further judgment of the characteristic action by the time range can effectively avoid the scoring error caused by the time deviation.

[0085] In step S3 of the application, because the experimental steps will set an expected time range for the characteristic action, the actual time of the characteristic action may deviate slightly from the prescribed time due to the actual situation of the operation of the experimental personnel, so a certain time deviation is allowed to prevent unfair or overly strict scoring due to too small time error, in step S2, if the characteristic action is judged to be in conformity with the prescribed action, but the time range is not in conformity, the loss of score is further judged according to the preset maximum tolerance time threshold, and then it is further judged that the characteristic action conforms to the prescribed action, but the time range of the characteristic action does not conform to the time range of the prescribed action, the score of such a case is;

[0086] The maximum tolerance time Δt is defined max If the characteristic action belongs to the prescribed action, the actual time Δt of the characteristic action exceeds the time ΔT of the prescribed action, and the actual time Δt of the characteristic action is less than or equal to the maximum tolerance time Δt max , it is summarized in the score loss table, and the score loss is calculated.

[0087] The time error is calculated based on the actual time Δt and the time ΔT of the prescribed action, and the calculation of the score loss includes ΔT-Δt is the time error;

[0088] If the time error is within the maximum tolerance time, the deduction ratio is Δt max -ΔT, the score is 1-deduction ratio, if the time error exceeds the maximum tolerance time, the score is 0.

[0089] In step S3, by setting the maximum tolerance time threshold, the score is dynamically calculated and the score loss is adjusted according to the difference between the actual time and the prescribed time, the adaptability and intelligent level of the characteristic action scoring mechanism are improved, the fairness of the scoring is ensured, the scoring difference caused by small deviation in time is avoided, and the uniformity of the scoring mechanism is realized.

[0090] In the step S3, if d is greater than the preset threshold value, it is considered that the characteristic action does not match the prescribed action, the characteristic action does not conform to the prescribed action, and it is determined that the characteristic action cannot be scored.

[0091] In the step S3, the characteristic action is judged according to the judgment standard of the step S2, if the characteristic action has been determined to conform to the prescribed action, on the premise, it is necessary to determine whether the step in which the characteristic action is located belongs to the sequential step, if not, it is indicated that the step still belongs to the preceding step, it is determined that the time of the characteristic action is less than the time of the prescribed action, it is summarized in the scoring category, and the operation of the steps S2 to S3 is performed.

[0092] If the step in which the characteristic action is located belongs to the sequential step, it is determined whether the preceding step of the sequential step is completed, because the preceding step of the sequential step can include a plurality of independent characteristic actions, if it is completed, and the time of the characteristic action in the sequential step is less than the time of the prescribed action, it is summarized in the scoring category, and the steps S2 to S3 are performed, if the preceding step has not been completed, the experimenter is prompted, and it is determined that the sequential step cannot be scored.

[0093] In the chemical experiment, some operation steps must rely on the completion result of the preceding step, for example, in some experiments, the preparation of the reaction, the addition of the chemical reagent, the performance of the mixing process, and even the control of the reaction time are all interrelated; if the preceding step is not completed, the operation of the subsequent sequential step can cause the reaction to fail, the data to be inaccurate, and even cause an experimental accident;

[0094] Therefore, by determining whether the preceding step of the sequential step is completed, it is ensured that the operation of the subsequent step is performed under reasonable conditions, if the preceding step has not been completed, the experimenter is prompted when the sequential step is performed, and it is determined that the step cannot be scored, which not only ensures the sequential nature of the experiment, but also avoids logical errors in the experimental steps.

[0095] The sequential step must rely on the completion of the preceding step, which avoids the experimenter skipping some steps or performing the experiment according to the operation sequence, avoids dangerous situations in the experiment, ensures the delicacy of the scoring mechanism, and improves the fairness and rationality of the scoring.

[0096] In the step S4, according to the completed action set obtained in the steps S2 and S3, and a series of prescribed actions defined in the experimental process, the prescribed action is taken as the experimental prescribed action set, by comparing the experimental prescribed action set and the completed action set, the omitted prescribed action is obtained, and the completeness of the step is calculated.

[0097] In summary, by dividing the several steps of the experimental process into the pre-step set and the sequential step set according to the sequence relationship, and extracting the characteristic actions of the experimental personnel, comparing the characteristic actions with the specified actions in the pre-step and the sequential step respectively, scoring according to whether the characteristic actions conform to the specified actions, ensuring that the operation of the experimental personnel conforms to the specification, when the characteristic actions conform to the specified actions, further dynamically scoring according to the time range and the sequence relationship of the actions, through the phased judgment, not only the recognition ability of the complex experimental operation can be improved, but also the fineness and accuracy of the scoring are ensured, and meanwhile, the several steps can be accurately monitored, preventing omissions or misoperations, and improving the fault tolerance and adaptability of the scoring.

[0098] The above is based on the ideal embodiment of the application, through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the application. The technical scope of the application is not limited to the content in the specification, and the technical scope must be determined according to the scope of claims.

Claims

1. An automatic scoring and optimization method for experimental steps based on image recognition, characterized in that, Includes the following steps: Step S1: Decompose the experimental process into a set of temporally related pre-steps and a set of sequential steps, each step containing a prescribed action with spatiotemporal constraint parameters; Step S2: Identify several characteristic actions of the experimenters, calculate the matching degree between each characteristic action and the prescribed action, and determine the action as a compliant action if it meets the threshold condition and mark it as a pre-scoring item; Step S3: Determine whether the compliant action belongs to the set of preceding steps or the set of sequential steps, obtain the action time of the compliant action, and dynamically calculate the score loss based on the judgment result and the spatiotemporal constraint parameters of the prescribed action corresponding to the compliant action. Step S4: Based on the judgment result, calculate the completeness of the set of preceding steps or the set of sequential steps, output the unrecognized prescribed actions, and score them; Step S5: Combine the scores from steps S3 and S4 to obtain the final score and step completeness.

2. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S1, based on the temporal correlation of the experimental steps, several steps of the experimental process are divided into a set of prerequisite steps and a set of sequential steps. Prerequisite steps are the prerequisites for sequential steps, and there are several parallel operations in the prerequisite steps. Each step includes a prescribed action with spatiotemporal constraints. Specifically, the preceding steps and sequential steps are divided into several characteristic actions, which are used as prescribed actions. The start and end anchor points of each characteristic action are defined, and the time points of the start and end anchor points are recorded as the time range of the characteristic action.

3. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S2, both the characteristic action and the prescribed action include the movement trajectory of the equipment.

4. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: Step S2 includes the following sub-steps: Step S21: Establish a unified spatiotemporal coordinate system and obtain the hand movements of the equipment movement trajectory in the characteristic movements, including the positions of key points of the wrist, fingertips, and elbow. Step S22: Record the time points of adjacent key points; Step S23: Based on real-time tracking and recording of key point position changes, and recording the positions of all key points as feature actions, using formula A... i (T)={(x1,y1,z1), (x2,y2,z2),..., (x i ,y i ,z i The expression is as follows:} where A i (T) represents the position of all keypoints at time t, (x) represents the position of all keypoints at time t i ,y i ,z i () indicates the position of a key point at time t; Step S24: Compare the feature action with the preset prescribed action to determine whether the feature action belongs to the prescribed action.

5. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 4, characterized in that: In step S21, identifying several characteristic actions of the experimenter includes the following steps: Step S211: Acquire several two-dimensional images of personnel movements and equipment status; Step S212: Convert several two-dimensional images into three-dimensional spatial coordinates, and select the center of the experimental platform as the origin of the coordinate system; Step S213: Map personnel movements and equipment positions to a coordinate system and record the position of equipment in the image data in real time.

6. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 4, characterized in that: Step S24 includes defining the position A' of the key point in the specified action at time t. i (T)={(x'1,y'1,z'1),(x'2,y'2,z'2),...,(x' t ,y' t ,z' t )}, calculate the direct distance between the actual key point and the prescribed action at time t. Where (x) i ,y i ,z i ) and (x' t ,y' t ,z' t (i) represents the spatial coordinates of the characteristic action and the prescribed action at time i. d is the Euclidean distance. A threshold is set. If d is less than or equal to the preset threshold, the feature action is considered to match the specified action and conform to the specified action. If d is greater than the preset threshold, the feature action is considered to fail to conform to the specified action.

7. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S3, if d is less than or equal to a preset threshold, the feature action is considered to match the specified action and conform to the specified action. It is then determined whether the step to which this feature action belongs is a set of preceding steps. If a set of preceding steps exists, the time of this feature action is further compared with the time of the specified action. Define the start time of each feature action as Ti and the end time as Ty. Then the time of the specified action is ΔT = ||Ty-Ti||. Record the start time and end time of the feature action, calculate the actual time Δt, and compare the actual time Δt with the time of the specified action. If the actual time is within the specified time range, the feature action is recorded as a score. If the feature action is a prescribed action and the feature action is a sequential step, then it is determined whether the preceding steps of this sequential step have been completed. If they have been completed and the time of the feature action in the sequential step is less than the time of the prescribed action, then it is included in the scoring row and column. Based on the judgment result and the spatiotemporal constraint parameters of the prescribed action corresponding to the compliant action, the score loss is dynamically calculated. If it has not been completed, then this sequential step is determined to receive no score.

8. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S3, the maximum tolerance time Δt is defined. max If the characteristic action is a prescribed action, and the actual time Δt of the characteristic action exceeds the time ΔT of the prescribed action, and the actual time Δt of the characteristic action is less than or equal to the maximum tolerable time Δt. max If the score is lost, it is categorized into the score loss category, and its score loss is calculated.

9. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S3, the time error is calculated based on the actual time Δt and the time ΔT of the prescribed action. The calculation of the score loss includes... ΔT-Δt represents the time error; If the time error is within the maximum tolerance time, the deduction ratio is Δt. max -ΔT, the score is 1 - the deduction ratio. If the time error exceeds the maximum tolerance time, the score is 0.

10. The automatic scoring and optimization method for experimental steps based on image recognition according to claim 1, characterized in that: In step S3, if d is greater than a preset threshold, the feature action is considered to be mismatched with the specified action, and the feature action does not meet the specified action requirements, so the feature action is judged to receive no points.

Images