Compensation method for testing experimental glassware

Abstract

The application discloses an experimental glassware detection compensation method, and specifically comprises the following steps: S1 target detection: obtaining a transparent glassware image video under an experimental environment; S2 video frame extraction: extracting a video frame image by calculating the confidence degree of each image in the image video of the step S1 containing the transparent glassware; S3 video frame image background preprocessing: extracting the subject color of the video frame image in the step S2, replacing the extracted subject color with a light gray color, and then performing a gray-scale processing on the video frame image; S4 transparent glassware target supplementing: detecting the transparent target in the video frame image processed in the step S3, compensating the coordinate information of the transparent target, and obtaining a transparent target contour. Based on the video frame extraction containing the confidence degree of the transparent glassware, the execution efficiency is ensured, the time consumption problem caused by the compensation method is relieved, and the accuracy of the transparent glassware detection in the experiment can be improved under a complex experimental background.

Description

Technical Field

[0001] This invention belongs to the field of computer image processing technology, and specifically relates to a method for testing and compensating experimental glassware. Background Technology

[0002] With the widespread application of artificial intelligence (AI) technology, various excellent neural networks have emerged in the field of computer vision. AI technology not only performs exceptionally well in traditional image processing but has also made comprehensive progress in deeper video analysis. For example, an automatic grading system for middle school physics, chemistry, and biology experiments uses video analysis technology to process the video content of the experiments and provide reasonable judgments. Currently, the grading methods of most experiment exam systems rely heavily on the precise positioning of the target equipment. Physics, chemistry, and biology experiments often involve the use of glassware such as test tubes, beakers, and gas collecting bottles. Glassware has high transparency, and different primary colors of background will produce different reflection colors in transparent equipment. Its color characteristics are easily affected by environmental factors such as the experimenter's clothing, laboratory background, and lighting.

[0003] Therefore, diverse experimental environments not only affect the extraction of color features from transparent equipment by target detection algorithms, but also weaken their ability to extract contour features, resulting in low accuracy for common target detection algorithms in detecting transparent glass equipment. Furthermore, the judgment methods for many scoring points in grading systems are closely related to the accurate detection of transparent equipment; the low detection rate of transparent equipment severely impacts the accuracy of grading system scoring. Existing methods mostly focus on improving the detection accuracy of transparent equipment, but the results have been minimal, and the detection of transparent equipment remains a technological barrier in the industry. Therefore, while improving the detection rate of transparent equipment, compensatory measures are particularly important, as they can effectively improve the accuracy of grading system judgments.

[0004] Chinese patent document CN113657770A discloses an experimental operation scoring system and method, but it rarely addresses the compensation for the detection of experimental glassware. Therefore, it is necessary to propose a method for compensating for the detection of experimental glassware to improve the accuracy of transparent glassware detection in the problem of misjudgment in the grading system. Summary of the Invention

[0005] The technical problem to be solved by this invention is to propose a method for testing and compensating for experimental glass instruments, which can improve the accuracy of testing transparent glass instruments in complex experimental settings.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for testing and compensating experimental glassware, specifically including the following steps:

[0007] S1 Target Detection: Acquiring images and videos of transparent glass equipment under experimental conditions;

[0008] S2 Video Frame Extraction: By calculating the confidence level that each frame in the image video of step S1 contains transparent glass equipment, video frame images are extracted.

[0009] S3 Video frame image background preprocessing: Extract the theme color of the video frame image in step S2, replace the extracted theme color with light gray, and then perform grayscale processing on the video frame image.

[0010] S4 Transparent Glass Equipment Target Supplement: Detect transparent targets in the video frame image processed by step S3, compensate for the coordinate information of the transparent targets, and obtain the outline of the transparent targets.

[0011] The detection of transparent glass instruments is easily affected by the experimental environment, and the low accuracy of transparent glass instrument detection can lead to misjudgment of scoring points. In the above technical solution, video frame extraction based on the confidence level of transparent glass instruments is used to ensure the execution efficiency of grading and scoring, which can alleviate the time consumption problem caused by the compensation method. Furthermore, the background preprocessing of the video frame image can alleviate the impact of the experimental environment on the detection of transparent glass instruments. This invention can improve the accuracy of transparent glass instrument detection in experiments under complex experimental backgrounds.

[0012] Preferably, the experimental glassware testing compensation method of the present invention further includes the following steps:

[0013] S5 positioning error judgment conditions: Based on the experimental score points, list the judgment conditions and the experimental equipment required for each judgment condition to form a spare list of experimental equipment; extract the corresponding detection target result list of the score points; compare the spare list of experimental equipment with the detection target result list of the same score point; discover the loss of test tube detection in a certain operation;

[0014] S6 analyzes adjacent conditions: it uses the state flags of adjacent scoring conditions to infer the actual state of the current scoring condition; that is, it infers the previous operation step of a certain step; that is, it performs frame difference inference.

[0015] S7 detects experimental phenomena: It detects experimental phenomena, combines the positional relationships of each target, infers and updates the scoring condition status markers; that is, it pushes the experimental phenomena.

[0016] The frame difference method is used to compare the experimental equipment spare list with the detection target result list to update the status flag of the judgment condition. The status flag of the corresponding judgment condition is updated by detecting the experimental phenomenon.

[0017] Preferably, the specific steps of video frame extraction in step S2 are as follows:

[0018] S21: List the glassware used in the experiment (L1);

[0019] S22: Use a one-stage object detection algorithm to perform frame-by-frame detection on the video and determine whether the video frame contains the category in L1.

[0020] S23: Calculate the confidence level of each video frame containing the L1 category based on the detection results;

[0021] S24: Extract video frames with a confidence level greater than 0.6 as the sequence to be processed, denoted as seq1.

[0022] Preferably, the specific steps of step S3 are as follows:

[0023] S31: Extract the main color from the seq1 image obtained in step S24. Read each frame of the image using the RGB color mode. Use the three RGB color channels as the three dimensions of the three-dimensional space to form a color space. Then project each pixel in the image into the color space. Next, map the coordinate points in the color space onto one-dimensional coordinates to obtain the quantity of each color. Then use a clustering algorithm to cluster the colors in the image to extract the main color of the image.

[0024] S32: Replace the theme color extracted in step S31 with light gray to match the background color of the environment where the glass equipment is located.

[0025] S33: Convert the color of the image from a three-dimensional RGB representation to a one-dimensional representation, then expand the gray values ​​with a large number of pixels in the image, and merge the gray values ​​with a small number of pixels. The processed image is denoted as img1.

[0026] Enhance image contrast to highlight the outline of transparent glass equipment.

[0027] Preferably, in step S33, the image grayscale processing is performed using the average value method and / or floating-point algorithm and / or integer method.

[0028] Preferably, the specific steps for supplementing the transparent glass instrument target in step S4 are as follows:

[0029] S41: Use the one-stage target detection model to detect the transparent target in img1 again. If the category in L1 is detected, add the new coordinates of the transparent target to the detection result list.

[0030] S42: Enable the side camera and capture the detection results of seq1 under the side camera. As long as the category in L1 is detected under one of the cameras, it indicates that this frame contains transparent glass equipment.

[0031] S43: Estimate the coordinates of the transparent equipment by using the coordinate information of the components of the transparent glass equipment; S44: Estimate the outline of the transparent target by using the relative position of the non-transparent equipment related to the transparent equipment.

[0032] Preferably, the specific steps for locating the misjudgment condition in step S5 are as follows:

[0033] S51: List the scoring criteria and the experimental equipment required for each scoring criterion to form a spare list of experimental equipment.

[0034] S52: Extract the list of detection target results corresponding to the sub-points;

[0035] S53: Comparing the spare list of experimental equipment with the list of detection target results for the same scoring point, it was found that the detection was lost, so the status of this scoring condition was marked as 0.

[0036] Preferably, the specific steps of step S6, which analyzes adjacent conditions, are as follows:

[0037] S61: Compare the spare experimental equipment list with the test target result list to check whether the equipment targets for each scoring condition are complete;

[0038] S62: Call the target detection algorithm to obtain the coordinate information of each target in the target detection result list, and then define the device state corresponding to the experimental step by combining the positional relationship between different target boxes;

[0039] S63: Combine the list of detection target results with the status markers of adjacent scoring conditions to infer the status marker of the current scoring condition.

[0040] Preferably, the specific steps for detecting the experimental phenomenon in step S7 are as follows:

[0041] S71: Definition of experimental phenomena and required conditions (reference table);

[0042] S72: Select a target detection model to detect the experimental phenomena;

[0043] S73: Based on the positional relationships of each target, infer and update the scoring condition status markers.

[0044] The experimental glassware detection compensation method of the present invention combines traditional image processing technology with reverse algorithm, which can solve the problem of misjudgment of scoring points caused by the loss or incorrect identification of glassware in complex experimental backgrounds. Attached Figure Description

[0045] Figure 1 This is an overall flowchart of the experimental glassware testing and compensation method of the present invention;

[0046] Figure 2This is a schematic diagram of the test tube fixing device in step S4 of the experimental glassware testing and compensation method of the present invention;

[0047] Figure 3 This is a schematic diagram of pouring alkaline solution into the test tube in step S7 of the experimental glassware testing and compensation method of the present invention;

[0048] Figure 4 This is a schematic diagram of adding colorless phenolphthalein solution to a test tube in step S7 of the experimental glassware testing and compensation method of the present invention. Detailed Implementation

[0049] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings.

[0050] Example: Figure 1 As shown, the experimental glassware testing compensation method in this embodiment specifically includes the following steps:

[0051] S1 Target Detection: Acquiring images and videos of transparent glass equipment under experimental conditions;

[0052] S2 Video Frame Extraction: By calculating the confidence level that each frame in the image video of step S1 contains transparent glass equipment, video frame images are extracted.

[0053] S3 Video Frame Image Background Preprocessing: Extract the main color of the video frame image in step S2, replace the extracted main color with light gray, and then perform grayscale processing on the video frame image; in this step, main color extraction, grayscale processing and histogram equalization can be used.

[0054] S4 Transparent Glass Equipment Target Supplement: Detect transparent targets in the video frame image processed in step S3, compensate for the coordinate information of the transparent targets, and obtain the outline of the transparent targets. In this step, existing target detection algorithms can be used for image detection of the side camera video frame, and then new coordinate information of the transparent targets can be calculated as compensation based on the detection results of the targets.

[0055] It also includes the following steps:

[0056] S5 Positioning Misjudgment Conditions: Based on the experimental scoring points, list the scoring conditions and the experimental equipment required for each condition to form a spare list of experimental equipment; extract the corresponding target result list for each scoring point; compare the spare list of experimental equipment with the target result list for the same scoring point;

[0057] S6 analyzes adjacent conditions: infers the actual state of the current scoring condition by using the state flags of adjacent scoring conditions;

[0058] S7 detects experimental phenomena: It detects experimental phenomena, combines the positional relationships of each target, and infers and updates the scoring condition status markers.

[0059] The specific steps of video frame extraction in step S2 are as follows:

[0060] S21: List the glassware used in the experiment (L1);

[0061] S22: Use a one-stage object detection algorithm to perform frame-by-frame detection on the video and determine whether the video frame contains the category in L1.

[0062] S23: Calculate the confidence level of each video frame containing the L1 category based on the detection results;

[0063] S24: Extract video frames with a confidence level greater than 0.6 as the sequence to be processed, denoted as seq1.

[0064] The specific steps of step S3 are as follows:

[0065] S31: Extract the main color from the seq1 image obtained in step S24. Read each frame of the image using the RGB color mode. Use the three RGB color channels as the three dimensions of the three-dimensional space to form a color space. Then project each pixel in the image into the color space. Next, map the coordinate points in the color space onto one-dimensional coordinates to obtain the quantity of each color. Then use a clustering algorithm to cluster the colors in the image to extract the main color of the image.

[0066] S32: Replace the theme color extracted in step S31 with light gray to match the background color of the environment where the glass equipment is located.

[0067] S33: Convert the color of the image from a three-dimensional RGB representation to a one-dimensional representation, then expand the gray values ​​with a large number of pixels in the image, and merge the gray values ​​with a small number of pixels. The processed image is denoted as img1.

[0068] In step S33, the image grayscale is processed using the average value method and / or floating-point algorithm and / or integer method.

[0069] The specific steps for supplementing the transparent glass equipment target in step S4 are as follows:

[0070] S41: Use the one-stage target detection model to detect the transparent target in img1 again. If the category in L1 is detected, add the new coordinates of the transparent target to the detection result list.

[0071] S42: Enable the side camera and capture the detection results of seq1 under the side camera. As long as the category in L1 is detected under one of the cameras, it indicates that this frame contains transparent glass equipment.

[0072] S43: Estimate the coordinates of the transparent equipment by using the coordinate information of the components of the transparent glass equipment; S44: Estimate the outline of the transparent target by using the relative position of the non-transparent equipment related to the transparent equipment.

[0073] Taking a dropper as an example, the detection model detected the coordinates (x1, y1, x2, y2) of the red rubber bulb, but not the coordinates of the dropper itself. First, based on the coordinates of the red rubber bulb, the height and width of the bulb, and its aspect ratio (ratio = height / width), are calculated. The range of this ratio defines whether the dropper is in a vertical position. In a vertical position, the dropper's width is similar to the rubber bulb's width, while the dropper's length is approximately 4 to 5 times the rubber bulb's length. This is used to estimate the dropper's coordinates. Finally, the outline of the transparent target is deduced by the relative positions of the non-transparent equipment related to the transparent equipment. For example, in the oxygen production experiment, the fixed test tube operation is shown in the corresponding schematic diagram. Figure 2 As shown, the detection model acquired the coordinate information of the cotton, rubber stopper, and potassium permanganate powder, but did not detect the test tube. Figure 2 As can be seen, the rubber stopper and potassium permanganate powder are located at opposite ends of the test tube. Combined with the position of the cotton, the outline of the test tube can be roughly drawn.

[0074] The specific steps for locating the misjudgment condition in step S5 are as follows:

[0075] S51: List the scoring criteria and the experimental equipment required for each scoring criterion to form a spare list of experimental equipment.

[0076] S52: Extract the list of detection target results corresponding to the sub-points;

[0077] S53: Comparing the spare list of experimental equipment with the list of detection target results for the same scoring point, it was found that the detection was lost, so the status of this scoring condition was marked as 0.

[0078] Taking the scoring point of pouring solution into a test tube as an example, the list of spare experimental equipment is as follows: {"Pour solution into test tube":["Open solution reagent bottle":["Solution reagent bottle", "cap"],"Pour solution into test tube":["Solution reagent bottle", "Test tube", "cap"],"Close solution reagent bottle":["Solution reagent bottle", "Cap", "Test tube", "Liquid surface"]}, and the list of detection target results is as follows: {"Pour solution into test tube":["Open solution reagent bottle":["Solution reagent bottle", "Cap"],"Pour solution into test tube":["Solution reagent bottle", "Cap"],"Close solution reagent bottle":["Solution reagent bottle", "Cap", "Test tube", "Liquid surface"]}; Finally, comparing the list of spare experimental equipment and the list of detection target results for the same scoring point, it is found that the test tube detection is lost in the "Pour solution into test tube" operation, causing the status of this scoring condition to be marked as 0.

[0079] The specific steps for analyzing adjacent conditions in step S6 are as follows:

[0080] S61: Compare the spare experimental equipment list with the test target result list to check whether the equipment targets for each scoring condition are complete;

[0081] S62: Call the target detection algorithm to obtain the coordinate information of each target in the target detection result list, and then define the device state corresponding to the experimental step by combining the positional relationship between different target boxes;

[0082] S63: Combine the list of detection target results with the status markers of adjacent scoring conditions to infer the status marker of the current scoring condition.

[0083] Taking a balance weighing operation as an example, firstly, it is determined whether targets such as two balance pans, two pieces of paper, weights, and the object to be weighed are detected. If there is a piece of paper on each of the two pans, with the solid to be measured on one side of the paper and a weight on the other side, it indicates that this frame meets the device state corresponding to the weighing action, and the corresponding step state mark is recorded as 1. Finally, the current scoring condition state mark is inferred by combining the list of detected target results with the state marks of adjacent scoring conditions. Similarly, taking the operation of pouring solution into a test tube as an example, after comparing the list and analyzing the target position relationship, the state marks of the scoring conditions "opening the solution reagent bottle" and "closing the solution reagent bottle" are both 1, and the liquid surface target in the target result list of this scoring point is located at the bottom of the test tube, inferring that the solution pouring operation was performed before closing the bottle cap. Therefore, the state mark of the condition "pouring solution into the test tube" is updated to 1.

[0084] The specific steps for detecting the experimental phenomena in step S7 are as follows:

[0085] S71: Definition of experimental phenomena and required conditions (reference table);

[0086] S72: Select a target detection model to detect the experimental phenomena;

[0087] S73: Based on the positional relationships of each target, infer and update the scoring condition status markers.

[0088] Taking the reaction experiment of alkali and indicator as an example, the following comparison table is defined: {"The liquid in the test tube turns red": ["Add alkali solution to the test tube", "Add colorless phenolphthalein solution to the test tube", "Shake the test tube until a red liquid appears in the test tube"]}. The list of detection target results obtained by the detection model is as follows: {"Alkali reacts with indicator": ["Add alkali solution to the test tube": ["Alkali solution reagent bottle", "Cork", "Test tube"], "Add colorless phenolphthalein solution to the test tube": ["Colorless phenolphthalein reagent bottle", "Dropper"], "Shake the test tube until a red liquid appears in the test tube": ["Test tube", "Red liquid"]}. Figure 3 and Figure 4 Show diagrams illustrating the operations of "adding an alkaline solution to a test tube" and "adding colorless phenolphthalein to a test tube," respectively. Figure 4 The list of test results reveals that the test tube was lost during the operation of "adding colorless phenolphthalein solution to the test tube," causing this scoring condition to be marked as 0. Firstly, as... Figure 3 As shown, the cork is placed upside down on the table, to the left of the alkaline solution bottle, indicating that the alkaline solution bottle is correctly opened. Then, since the alkaline solution bottle is located at the upper left of the test tube, and the relative positions of the bottle mouth and test tube top are the same as the relative positions of the alkaline solution bottle (NaOH) on the test tube (test_tube), the scoring condition of "adding alkaline solution to the test tube" is satisfied. Next, from... Figure 4 The detection results list shows that the model detected red liquid, and the red liquid's coordinates are located at the bottom of the test tube. This indicates that the scoring condition of "shaking the test tube until red liquid appears in the test tube" is met, and the experimental phenomenon of alkali reacting with colorless phenolphthalein to produce a red color has occurred. Therefore, it can be deduced that the operation of "adding alkali solution to the test tube" was completed after "adding colorless phenolphthalein solution to the test tube". Thus, the status flag of this scoring condition is updated to 1.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for testing and compensating laboratory glassware, characterized in that, Specifically, the following steps are included: S1 Target Detection: Acquiring images and videos of transparent glass equipment under experimental conditions; S2 Video Frame Extraction: By calculating the confidence level that each frame in the image video of step S1 contains transparent glass equipment, video frame images are extracted. S3 Video frame image background preprocessing: Extract the theme color of the video frame image in step S2, replace the extracted theme color with light gray, and then perform grayscale processing on the video frame image. S4 Transparent Glass Equipment Target Supplement: Detect transparent targets in the video frame image processed by step S3, compensate for the coordinate information of the transparent targets, and obtain the outline of the transparent targets; The specific steps of step S3 are as follows: S31: Extract the main color from the seq1 image obtained in step S24. Read each frame of the image using the RGB color mode. Use the three RGB color channels as the three dimensions of the three-dimensional space to form a color space. Then project each pixel in the image into the color space. Next, map the coordinate points in the color space onto one-dimensional coordinates to obtain the quantity of each color. Then use a clustering algorithm to cluster the colors in the image to extract the main color of the image. S32: Replace the theme color extracted in step S31 with light gray to match the background color of the environment where the glass equipment is located. S33: Convert the color of the image from a three-dimensional RGB representation to a one-dimensional representation, then expand the gray values ​​with a large number of pixels in the image, and merge the gray values ​​with a small number of pixels. The processed image is denoted as: img1; The specific steps for supplementing the transparent glass equipment target in step S4 are as follows: S41: Use the one-stage target detection model to detect the transparent target in img1 again. If the category in L1 is detected, add the new coordinates of the transparent target to the detection result list. S42: Enable the side camera and capture the detection results of seq1 under the side camera. As long as the category in L1 is detected under one of the cameras, it indicates that this frame contains transparent glass equipment. S43: Zero-to-round estimation, that is, estimating the coordinates of transparent equipment by using the coordinate information of the components of transparent glass equipment; S44: Calculate the outline of a transparent target by the relative position of non-transparent equipment associated with the transparent equipment.

2. The experimental glassware testing and compensation method according to claim 1, characterized in that, It also includes the following steps: S5 Positioning Misjudgment Conditions: Based on the experimental scoring points, list the scoring conditions and the experimental equipment required for each condition to form a spare list of experimental equipment; extract the corresponding target result list for each scoring point; compare the spare list of experimental equipment with the target result list for the same scoring point; S6 analyzes adjacent conditions: infers the actual state of the current scoring condition by using the state flags of adjacent scoring conditions; S7 detects experimental phenomena: It detects experimental phenomena, combines the positional relationships of each target, and infers and updates the scoring condition status markers.

3. The experimental glassware testing and compensation method according to claim 1, characterized in that, The specific steps of video frame extraction in step S2 are as follows: S21: List the glassware used in the experiment (L1); S22: Use a one-stage object detection algorithm to perform frame-by-frame detection on the video and determine whether the video frame contains the category in L1. S23: Calculate the confidence level of each video frame containing the L1 category based on the detection results; S24: Extract video frames with a confidence level greater than 0.6 as the sequence to be processed, denoted as seq1.

4. The experimental glassware testing and compensation method according to claim 1, characterized in that, In step S33, the image grayscale is processed using the average value method and / or floating-point algorithm and / or integer method.

5. The experimental glassware testing and compensation method according to claim 2, characterized in that, The specific steps for locating the misjudgment condition in step S5 are as follows: S51: List the scoring criteria and the experimental equipment required for each scoring criterion to form a spare list of experimental equipment. S52: Extract the list of detection target results corresponding to the sub-points; S53: Comparing the spare list of experimental equipment with the list of detection target results for the same scoring point, it was found that the detection was lost, so the status of this scoring condition was marked as 0.

6. The experimental glassware testing and compensation method according to claim 5, characterized in that, The specific steps for analyzing adjacent conditions in step S6 are as follows: S61: Compare the spare experimental equipment list with the test target result list to check whether the equipment targets for each scoring condition are complete; S62: Call the target detection algorithm to obtain the coordinate information of each target in the target detection result list, and then define the device state corresponding to the experimental step by combining the positional relationship between different target boxes; S63: Combine the list of detection target results with the status markers of adjacent scoring conditions to infer the status marker of the current scoring condition.

7. The experimental glassware testing and compensation method according to claim 6, characterized in that, The specific steps for detecting the experimental phenomena in step S7 are as follows: S71: Definition of experimental phenomena and required conditions (reference table); S72: Select a target detection model to detect the experimental phenomena; S73: Based on the positional relationships of each target, infer and update the scoring condition status markers.

Images