A method for detecting the tilt angle of a narrow-mouth bottle for intelligent experiment evaluation
By using a key point regression model to detect key points at the mouth of reagent bottles, the problem of accuracy and computational resources in detecting the tilt angle of reagent bottles in existing technologies has been solved, thus achieving high-precision chemical experiment evaluation.
Patent Information
- Application Number
- CN202210729388.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-24
- Publication Date
- 2026-02-03
- Estimated Expiration
- 2042-06-24
AI Technical Summary
Existing technologies struggle to accurately detect the tilt angle of reagent bottles in chemical experiments, especially transparent ones, during intelligent experimental evaluation. Furthermore, semantic segmentation methods are resource-intensive, and target detection methods lack sufficient accuracy.
A keypoint regression model is adopted, which uses a feature extractor, an upsampling module, a convolutional neural network and a convolutional layer, combined with loss function optimization, to detect two key points at the mouth of the reagent bottle. A two-dimensional Gaussian kernel function is used to generate heat map labels, and the model is iteratively optimized. Finally, the tilt angle is calculated through the coordinates of the key points.
It enables precise detection of the tilt angle of reagent bottles, reduces the computational resource requirements, and improves detection accuracy and labeling efficiency.
Smart Images
Figure CN115187896B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent experimental evaluation, and in particular to a method for detecting the tilt angle of a narrow-mouthed bottle used in intelligent experimental evaluation. Background Technology
[0002] With the rise of deep learning, computer vision technology has been continuously developed. Among them, object detection technology, represented by object detection and instance segmentation, is particularly prominent and has been successfully implemented in various industries. Common intelligent assessment schemes are mostly based on object detection, judging the correctness of some experimental operations by the relative position and changes of the detection box. This invention is aimed at chemistry experiments in the physics, chemistry and biology experimental assessment of junior and senior high schools. In chemistry experiments, reagent bottles are usually used as commonly used reagent containers. Therefore, there are many related scoring points and operations around reagent bottles, such as pouring solutions by pouring and correctly transferring liquids. In order to more meticulously assess whether students' experimental actions are standardized in the assessment, it is necessary to detect the tilt angle of the reagent bottle during operation.
[0003] Regarding the tilt angle of reagent bottles during use, the following are some common methods:
[0004] 1. Directly based on target detection, and then roughly estimating the tilt angle of the reagent bottle according to the aspect ratio of the detection frame, this method can only make a rough estimate and cannot meet the requirements of experimental investigation.
[0005] 2. Semantic segmentation-based detection methods detect tilt angles based on the overall outline of the detected object. Compared to object detection methods that rely on the aspect ratio of the bounding box for rough judgment, semantic segmentation-based methods are relatively more accurate in determining tilt angles. However, since the detection equipment is transparent, outline information is easily lost. If there is a deviation in the overall outline detection, it will have a significant impact on the final tilt angle detection. At the same time, the annotation of semantic segmentation also requires a lot of manpower and resources. Summary of the Invention
[0006] The purpose of this invention is to provide a method for detecting the tilt angle of a narrow-mouthed bottle for intelligent experimental evaluation, so as to achieve accurate experimental evaluation while reducing the computing power of intelligent evaluation.
[0007] To address the aforementioned technical problems, this invention provides a method for detecting the tilt angle of a narrow-mouthed bottle for intelligent experimental evaluation, comprising the following methods:
[0008] When annotating / inferring relevant experimental evaluation videos, extract images containing reagent bottles;
[0009] Train the keypoint regression model and iteratively optimize it based on the loss function;
[0010] The picture of the extracted reagent bottle is input into a key point regression model to obtain two key point coordinates (x1, y1) and (x2, y2) at the mouth of the reagent bottle;
[0011] According to the formula
[0012] Further, the key point model comprises a feature extractor, two up-sampling modules, a first convolutional neural network, and a convolutional layer connected in sequence, the first convolutional neural network outputs a feature tensor result1, and the convolutional layer outputs a feature tensor result2.
[0013] Further, the specific steps for training the key point model are:
[0014] The picture of the reagent bottle is input into the key point model to obtain a feature tensor result1 and result2 with a channel number equal to the number of required key points;
[0015] After the result1 is processed by an activation function, the feature map of the first channel is used as a heat map heatmap1 of the first key point, and the feature map of the second channel is used as a heat map heatmap2 of the second key point;
[0016] The heat map heatmap1' and the heat map heatmap2' are generated by using a two-dimensional Gaussian kernel function as labels label;
[0017] According to the positive and negative sample indexes, the positive and negative samples are taken out to calculate the loss, and the model is iteratively optimized according to the final loss function.
[0018] Further, according to the positive and negative sample indexes, the positive and negative samples are taken out to calculate the loss, and the model is iteratively optimized according to the final loss function.
[0019] The region with a value greater than 0 in the heat map heatmap1' and the heat map heatmap2' is taken as a positive sample, and the indexes one_pos_mask and two_pos_mask of the positive sample in the feature vector matrix are obtained;
[0020] The region with a value equal to 0 in the heat map heatmap1' and the heat map heatmap2' is taken as a negative sample, and the indexes one_neg_mask and two_neg_mask of the negative sample in the feature vector matrix are obtained;
[0021] According to the positive and negative sample indexes, the positive and negative samples in the heat map heatmap1 and the heat map heatmap1' are taken out to calculate the loss, and the one_pos_loss and one_neg_loss are obtained, and similarly, the two_pos_loss and two_neg_loss are obtained from the heat map heatmap2 and the heat map heatmap2'.
[0022] Similarly, result2 takes the processing mode of result1.
[0023] Further, the loss function of the positive and negative samples is MSELoss.
[0024] Further, the final loss function is:
[0025] Loss=one_pos_loss+two_pos_loss+0.1*one_neg_loss+0.1*two_neg_loss;
[0026] one_pos_loss is the positive sample loss of result1, one_neg_loss is the negative sample loss of result1, two_pos_loss is the positive sample loss of result2, and two_neg_loss is the negative sample loss of result2.
[0027] Compared with the prior art, the present application has at least the following beneficial effects:
[0028] The present application proposes a key point detection scheme based on heatmap regression, which is followed by a target detection network, first detects the reagent bottle, and then locates the two key points of the reagent bottle mouth to judge the inclination angle, has the advantages of being beneficial to labeling, high detection precision, etc., can better target related chemical experiments for more accurate examination, teaching and evaluation, and combined with a lightweight backbone network, can greatly reduce the cost of computing power. BRIEF DESCRIPTION OF DRAWINGS
[0029] Figure 1 The flowchart of an embodiment of the narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation of the present application;
[0030] Figure 2 The overall structure diagram of the key point training model of an embodiment of the narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation of the present application;
[0031] Figure 3 The key point detection network structure diagram of an embodiment of the narrow-mouth bottle inclination angle detection method for intelligent experiment evaluation of the present application. DETAILED DESCRIPTION
[0032] The method for detecting the tilt angle of a narrow-mouth bottle for intelligent experiment evaluation will be described in more detail below in conjunction with the schematic drawings, in which the preferred embodiments of the present application are represented, it being understood that the person skilled in the art can modify the present application as described herein while still achieving the advantageous effects of the present application. Therefore, the following description should be understood as a broad knowledge for the person skilled in the art and not as a limitation of the present application.
[0033] The present application will be described in more detail in the following paragraphs with reference to the drawings, which are merely intended to illustrate the present application by way of example and are not intended to be limiting of the present application. The advantages and features of the present application will become more clearly apparent from the following description and claims. It should be noted that the drawings are in extremely simplified form and employ non-precise dimensions for the purpose of facilitating a convenient and clear illustration of the embodiments of the present application.
[0034] As shown in Figure 1 , the present embodiment proposes a method for detecting the tilt angle of a narrow-mouth bottle for intelligent experiment evaluation, comprising the following methods:
[0035] taking out the pictures containing the reagent bottle when labeling / reasoning the relevant experiment evaluation video;
[0036] training the key point regression model and iteratively optimizing the key point regression model according to the loss function;
[0037] sending the taken pictures containing the reagent bottle to the key point regression model to obtain the two key point coordinates (x1, y1) and (x2, y2) at the mouth of the reagent bottle;
[0038] according to the formula
[0039] The following preferred embodiments of the method for detecting the tilt angle of a narrow-mouth bottle for intelligent experiment evaluation are listed to clearly illustrate the content of the present application, it being understood that the content of the present application is not limited to the following embodiments, and other improvements by the conventional technical means of the person skilled in the art are also within the scope of the idea of the present application.
[0040] (1) taking out the pictures containing the reagent bottle when labeling / reasoning the relevant experiment evaluation video.
[0041] Specifically, the common intelligent evaluation scheme is mostly based on target detection, the correctness of some experimental operations is judged by the relative position and change of the detection frame, and the relevant experiment video is labeled / reasoned to take out the pictures containing the reagent bottle.
[0042] (2) training the key point regression model and iteratively optimizing the key point regression model according to the loss function.
[0043] Specifically, the marking of key points on the reagent bottle mainly refers to two key points at the bottle opening: point 1 on the left and point 2 on the right. The specific marking details are as follows: Figure 1 As shown, the label content is the coordinate value of the relevant key point. For the above label content, taking the width and height as W and H as an example, the key point coordinates are (x1, y1), then the label content is (x1 / W, y1 / H).
[0044] In one embodiment, (a) the overall structure of the keypoint training model is as follows: Figure 2 As shown, a lightweight backbone network is used as a feature extractor, followed by two DUC modules to upsample the feature map (the number of upsampling modules and the upsampling factor can be flexibly adjusted). Then, a convolutional neural network is used to obtain a feature tensor result1 with the number of channels equal to the required number of key points. This tensor is then concatted with the tensor generated by the second DUC module to achieve feature fusion. The fused features are then passed through a convolutional layer to obtain a feature tensor result2 with the number of channels equal to the required number of key points.
[0045] (b) During the training process, such as Figure Two The results shown are Result1 and Result2. Taking an input image size of 480*480 as an example, both Result1 and Result2 are 1*2*60*60 tensors (in the keypoint regression model above, if n keypoints need to be regressed, the generated tensor shape is 1*n*60*60). Result1 is processed with a sigmoid function. The feature map of the first channel serves as the heatmap1 for the first keypoint, with a shape of 1*1*60*60. Similarly, the second channel serves as the heatmap2 for the second keypoint. Based on the coordinate information in the labels, a two-dimensional Gaussian kernel function is used to generate heatmap1' and heatmap2' as labels. The regions with median values greater than 0 are considered positive samples, and their indices in the feature vector matrix are obtained as `one_pos_mask`. Regions with median values equal to 0 are considered negative samples, and their indices in the feature vector matrix are obtained as `one_neg_mask`. Similarly, in `heatmap2'`, indices `two_pos_mask` and `two_neg_mask` are obtained. Based on the positive and negative sample indices, we can extract the positive and negative samples from `heatmap1` and `heatmap1'` and apply the loss function (MSELoss) to obtain `one_pos_loss` and `one_neg_loss`, respectively. Similarly, `heatmap2` and `heatmap2'` yield `two_pos_loss` and `two_neg_loss`.
[0046] (c) Following the same operations as in step 2 for result1, perform the same operations on result2. Let the four losses obtained from result1 be one_pos_loss1, one_neg_loss1, two_pos_loss1, and two_neg_loss1, and the four losses obtained from result2 be one_pos_loss2, one_neg_loss2, two_pos_loss1, and two_neg_loss2. The losses are set as follows:
[0047] one_pos_loss=one_pos_loss1+2*one_pos_loss2
[0048] one_neg_loss=one_neg_loss1+2*one_neg_loss2
[0049] two_pos_loss=two_pos_loss1+2*two_pos_loss2
[0050] two_neg_loss=two_neg_loss1+2*two_neg_loss2
[0051] The final loss function is as follows:
[0052] Loss = one_pos_loss + two_pos_loss + 0.1*one_neg_loss + 0.1*two_neg_loss, and then iteratively optimize the model based on the above loss function.
[0053] (3) Input the extracted image containing the reagent bottle into the keypoint regression model to obtain the coordinates (x1, y1) and (x2, y2) of two keypoints at the mouth of the reagent bottle, and then apply the formula...
[0054] Specifically, relying on the existing target detection network, when a reagent bottle is detected, it is cut out according to the detection frame and sent to the aforementioned key point detection network (e.g., Figure 3 During the inference process shown, the network directly takes result2 as the result, which gives the coordinates of two points (x1, y1) and (x2, y2) at the mouth of the reagent bottle. The required tilt angle can be obtained by using the formula arctan((y2-y1) / (x2-x1)).
[0055] In summary, this invention proposes a key point detection scheme based on heatmap regression that follows the target detection network. It first detects the reagent bottle and then determines the tilt angle by locating two key points at the bottle opening. This scheme has advantages such as improved annotation and high detection accuracy, enabling more precise testing, teaching, and evaluation of related chemical experiments. Furthermore, combined with a lightweight backbone network, it can significantly reduce computational costs.
[0056] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for detecting the tilt angle of a narrow-mouthed bottle for intelligent experimental evaluation, characterized in that, Including the following methods: When annotating / inferring experimental evaluation videos, extract images containing reagent bottles; Train the keypoint regression model and iteratively optimize it based on the loss function; The extracted image containing the reagent bottle is fed into the keypoint regression model to obtain the coordinates of two keypoints (x1, y1) and (x2, y2) at the mouth of the reagent bottle. According to the formula Calculate the tilt angle of the reagent bottle; The key point regression model training includes: extracting positive and negative samples according to the positive and negative sample indices, applying loss to each sample, and iteratively optimizing the model based on the final loss function; The specific steps for training the keypoint regression model are as follows: The image containing the reagent bottle is input into the key point regression model to obtain feature tensors result1 and result2, which have the required number of channels for the number of key points. After applying the activation function to r esu lt 1, the feature map of the first channel is used as the heatmap1 of the first key point, and the feature map of the second channel is used as the heatmap2 of the second key point. Heatmap1' and heatmap2' are generated using a two-dimensional Gaussian kernel function and used as labels. Regions with values greater than 0 in both heatmap1' and heatmap2' are considered positive samples, while regions with values equal to 0 in both heatmap1' and heatmap2' are considered negative samples. Similarly, result2 will be processed in the same way as result1; The final loss function is: Loss=one_pos_loss+two_pos_loss+0.1×one_neg_loss+0.1×two_neg_loss; Where one_pos_loss is the positive sample loss of result1, one_neg_loss is the negative sample loss of result1, two_pos_loss is the positive sample loss of result2, and two_neg_loss is the negative sample loss of result2.
2. The method for detecting the tilt angle of a narrow-necked bottle for intelligent experimental evaluation as described in claim 1, characterized in that, The keypoint regression model includes a feature extractor, two upsampling modules, a first convolutional neural network, and a convolutional layer connected in sequence. The first convolutional neural network outputs a feature tensor result1, and the convolutional layer outputs a feature tensor result2.
3. The method for detecting the tilt angle of a narrow-necked bottle for intelligent experimental evaluation as described in claim 1, characterized in that, Based on the positive and negative sample indices, positive and negative samples are retrieved and loss is applied separately: The regions in heatmap1' and heatmap2' with values greater than 0 are taken as positive samples, and their indices one_pos_mask and two_pos_mask in the feature vector matrix are obtained. The regions in heatmap1' and heatmap2' with values equal to 0 are taken as negative samples, and their indices one_neg_mask and two_neg_mask in the feature vector matrix are obtained. Based on the positive and negative sample indices, positive and negative samples are extracted from heatmap1 and heatmap1', and losses are applied to them respectively to obtain one_pos_loss and one_neg_loss. Similarly, heatmap2 and heatmap2' yield two_pos_loss and two_neg_loss.
4. The method for detecting the tilt angle of a narrow-necked bottle for intelligent experimental evaluation as described in claim 3, characterized in that, The loss function for both positive and negative samples is MSELoss.