Container type identification method, device and system

Abstract

The invention relates to the technical field of medical treatment, in particular to a container type identification method, device and system, and the method comprises the steps: obtaining a sample container image obtained by photographing a sample container on a sample rack; inputting the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence coefficient of the sample container; inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence coefficient of the sample container; and determining a target container type in the first container type and the second container type according to the first confidence coefficient and the second confidence coefficient. Therefore, the problem of low sample container identification accuracy in the prior art can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and in particular to a container type identification method, device and system. Background Art

[0002] In today's medical diagnosis, biological experiments and pharmaceutical industries, sample container type identification is an indispensable task, especially in automated testing and sample management. Accurate and rapid identification of sample container types is essential to improving work efficiency and accuracy. However, current sample containers come in a wide variety of types, complex environments, and are obstructed.

[0003] In the prior art, some deep learning is used for automatic identification in sample container type recognition. Some image processing methods based on machine vision are used to identify the type of sample container. However, this method usually relies on specific algorithms and rules to extract and match features in the image.

[0004] However, due to the diversity and quality of training data, these methods often fail to achieve satisfactory recognition results and have low recognition accuracy when faced with sample containers of different shapes and sizes, and possibly with occlusion or noise. The recognition accuracy of sample containers in complex scenarios needs to be improved. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a container type identification method, device and system, which can effectively solve the problem of low sample container identification accuracy in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a container type identification method, comprising:

[0007] Acquire a sample container image obtained by photographing the sample container on the sample rack;

[0008] Inputting the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence level of the sample container;

[0009] Inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container;

[0010] A target container type is determined from among the first container type and the second container type according to the first confidence level and the second confidence level.

[0011] In some embodiments, it also includes:

[0012] Performing offset detection and tilt detection on the sample container image, determining whether the sample container image is within a detection range according to the detection results, and outputting image acquisition position or angle adjustment information of the sample container image.

[0013] In some embodiments, performing offset detection and tilt detection on the sample container image and determining whether the sample container image is within a detection range according to the detection results includes:

[0014] Extracting the central axis of the sample container image to obtain the target central axis; comparing the target central axis with a set standard central axis; if the horizontal deviation between the target central axis and the standard central axis is within a set threshold range, determining that the sample container image is within a horizontal detection range, and determining that the offset detection has passed;

[0015] Using edge detection and straight line detection to calculate the slope of the sample container in the sample container image; if the slope is within a preset slope range, confirming that the tilt detection has passed;

[0016] If it is confirmed that the offset detection has passed and the tilt detection has passed, then a prompt message indicating that the image acquisition position and angle are correct is output; otherwise, corresponding adjustment information is output according to the horizontal deviation or the slope.

[0017] In some embodiments, determining the target container type from the first container type and the second container type according to the first confidence level and the second confidence level includes:

[0018] If the first confidence level is greater than the second confidence level, the first container type is determined to be the target container type; otherwise, the second container type is determined to be the target container type.

[0019] In some embodiments, inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container includes:

[0020] The container detection network model performs semantic segmentation on the sample container image to obtain a target region of interest; and obtains a mask image containing the outline of the sample container according to the target region;

[0021] Extracting features from the mask image to obtain container features;

[0022] The container feature is matched with features of each known container type in a preset feature library to obtain the second container type and the second confidence level.

[0023] In some embodiments, at least one of the following is also included:

[0024] Item 1: The container classification network model is trained using the following method:

[0025] Using a preset plug-in to mark the region of interest in each image included in the first target image data set to obtain a first training data set;

[0026] Using the first training data set to train the constructed first neural network to obtain the container classification network model;

[0027] Item 2: The container detection network model is trained using the following method:

[0028] Using a preset plug-in to mark the regions of interest in each image included in the second target image data set, and outputting a mask image corresponding to each region of interest;

[0029] The labeled second target image data set and the corresponding mask images are used as the second training data set to train the constructed second neural network to obtain the container detection network model.

[0030] In some embodiments, the first neural network includes a first YOLOv8 network; the first YOLOv8 network includes a feature extraction backbone module and a regression classification module connected in sequence;

[0031] The second neural network includes a second YOLOv8 network; the second YOLOv8 network includes a feature extraction backbone module, a feature fusion module and a regression classification module connected in sequence.

[0032] In some embodiments, the container classification network model and the container detection network model both adopt a dynamic weight balancing loss function and a Focal Loss loss function during training; wherein the category weight in the dynamic weight balancing loss function is determined based on the first parameter in the Focal Loss loss function being less than a preset threshold as the target.

[0033] In a second aspect, an embodiment of the present application provides an experimental control device, the experimental control device comprising a processor and a memory, the memory storing a computer program, and the processor being used to execute the computer program to implement a container type identification method provided in the first aspect of the present application.

[0034] In a third aspect, an embodiment of the present application provides an experimental system, the experimental system comprising: a camera and an experimental control device;

[0035] The camera is used to take a picture of the sample container on the sample rack to obtain an image of the sample container;

[0036] The experiment control device is used to execute a container type recognition method provided in the second aspect of the present application on the sample container image.

[0037] The embodiments of the present application have the following beneficial effects:

[0038] The present application obtains a sample container image by photographing a sample container on a sample rack; inputs the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence level of the sample container; inputs the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container; and determines a target container type from the first container type and the first container type based on the first confidence level and the second confidence level. The present application uses a container classification network model and a container detection network model to recognize the same image, and the two results obtained are mutually verified to obtain a target container type. Therefore, the present application can effectively solve the problem of low accuracy in sample container recognition in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 An application scenario diagram of the experimental system of the embodiment of the present application is shown;

[0041] Figure 2 A schematic diagram showing a process of a method for identifying a container type according to an embodiment of the present application is shown;

[0042] Figure 3 Another schematic diagram of the process of the container type identification method according to an embodiment of the present application is shown;

[0043] Figure 4 A schematic diagram of a network structure of a first YOLOv8 network in the container type identification method according to an embodiment of the present application is shown;

[0044] Figure 5 A schematic diagram of the network structure of a second YOLOv8 network in the container type identification method in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0046] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0047] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0048] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0049] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0050] In order to improve the recognition accuracy of sample container types, the present application provides a container type recognition method, device and system. Among them, one technical point in the container type recognition method of the present application is to use deep learning technology to automatically recognize the sample container type while segmenting the sample container image, extracting container data, and on this basis, combining traditional algorithms to verify the conclusion, thereby improving the recognition accuracy.

[0051] like Figure 1 As shown, the present application provides an experimental system, which exemplarily includes: a camera and an experimental control device. The camera is used to take a picture of the sample container on the sample rack to obtain an image of the sample container.

[0052] The experimental control equipment is used to implement the container type identification method of the present application to identify the sample container on the sample rack. The experimental control equipment of the embodiment of the present application includes test automation control equipment, sample management equipment, medical diagnostic equipment, biological experiment control equipment, etc. Among them, the medical diagnostic equipment can be, for example, a chemiluminescence analyzer, a glycated hemoglobin analyzer, a blood analyzer, a specific protein analyzer, etc.

[0053] The container type identification method is described below in conjunction with some specific embodiments.

[0054] Figure 2 A flow chart of a method for identifying a container type according to an embodiment of the present application is shown. Exemplarily, the method for identifying a container type includes the following steps:

[0055] S100, acquiring a sample container image obtained by photographing the sample container on the sample rack. In the embodiment of the present application, the camera in the vision module photographs the sample container on the sample rack.

[0056] Furthermore, in order to improve the image quality and expand the scope of application, the sample container image obtained by photographing the sample container on the sample rack includes:

[0057] The obtained initial image of the sample container is preprocessed to obtain a sample container image; the preprocessing includes: at least one of contrast change, saturation adjustment, hue shift, simulated lighting conditions and camera white balance. The simulated lighting conditions include adding Gaussian noise, salt and pepper noise, etc.

[0058] Among them, the brightness, contrast, saturation, etc. of the image are adjusted to improve the image quality. Some noise (such as Gaussian noise and salt and pepper noise) is added to simulate different lighting conditions.

[0059] S200: Input the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence level of the sample container.

[0060] In one implementation, the container classification network model is trained using the following method:

[0061] S211, using a preset plug-in to mark the region of interest in each image included in the first target image data set to obtain a first training data set.

[0062] For example, the Labelme plug-in is used to perform instance segmentation and annotation on the collected sample container images, specifically annotate the type of sample container, and provide label data for training the classification network model.

[0063] S212, using the first training data set to train the constructed first neural network to obtain a container classification network model. For example, if the first neural network uses a convolutional neural network (CNN), the training and recognition process of the first neural network includes the following steps:

[0064] Data preparation: Collect a large number of sample container images and annotate each sample container image to obtain a training data set. Use the camera or scanner in the vision module to take pictures of the sample containers in the sample rack to obtain a large number of sample container images.

[0065] Model training: The convolutional neural network (CNN) constructed by training the labeled training data set is used to learn the characteristics of different types of sample containers and obtain a container classification network model. It can be understood that the sample container image also needs to be preprocessed before labeling the sample container image, and the preprocessing includes contrast change, saturation adjustment, hue shift, adding Gaussian noise, salt and pepper noise, etc., simulating at least one of the lighting conditions and camera white balance.

[0066] Feature extraction: In the recognition stage, the sample container image to be identified is input into the trained convolutional neural network CNN (corresponding to the container classification network model), and the trained convolutional neural network CNN will automatically extract the features in the sample container image.

[0067] Classification and recognition: Based on the extracted features, the convolutional neural network CNN classifies the sample container image into the corresponding sample container type. That is, the convolutional neural network CNN outputs the first container type and the first confidence of the sample container.

[0068] It can be understood that the sample container image is input into the container classification network model for feature recognition to obtain the first container type and the first confidence of the sample container, including:

[0069] S300: Input the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container.

[0070] In one implementation, the container detection network model is trained using the following method:

[0071] S311, using a preset plug-in to mark the regions of interest in each image included in the second target image data set, and outputting mask images corresponding to each region of interest.

[0072] S312: Use the labeled second target image data set and the corresponding mask images as a second training data set to train the constructed second neural network to obtain a container detection network model.

[0073] It can be understood that the sample container image is input into the container detection network model for semantic segmentation and feature recognition to obtain the second container type and the second confidence of the sample container, including:

[0074] S321, the container detection network model performs semantic segmentation on the sample container image to obtain the target region of interest; and obtains a mask image containing the outline of the sample container according to the target region. For example, the YOLOv8 algorithm is used to perform target detection and semantic segmentation on the target region, identify the target region of interest, and create a mask outline image (mask image) of the target of interest according to its semantic segmentation result.

[0075] S322, extracting features from the mask image to obtain container features.

[0076] S323, matching the container feature with features of each known container type in a preset feature library to obtain a second container type and a second confidence level.

[0077] The training and prediction process of the container detection network model includes:

[0078] Data preparation: Collect a large number of sample container images and annotate each sample container image.

[0079] Model segmentation: Use the adopted deep learning model to directly segment the container part in the sample container image to obtain mask information.

[0080] Mask feature extraction: Extract the feature value of the sample container in the mask information, compare it in the preset feature library, and obtain the identified container type. In the preset feature library, set the range of mask information of different known types of containers, and obtain the container type result by comparing with each range.

[0081] Result verification: The results of the container detection network model identification are used to verify the output results of the container classification network model.

[0082] This application segments the area of ​​the input sample container image, divides and crops the region of interest in the sample container area involved, and reduces the interference information and pixels input into the container detection network model. In order not to lose resolution and to be able to train at smaller pixels, this application first adjusts the position of the camera in the visual module on the sample rack, as well as the segmented ROI area (region of interest), and sets the size of the region of interest to 320*640, and then formats it into a 320*320 image as a training set.

[0083] S400: Determine a target container type from a first container type and a second container type according to a first confidence level and a second confidence level. Figure 3 As shown, a decision module is also included, and the decision module determines the target container type from the first container type and the second container type according to the first confidence level and the second confidence level.

[0084] It can be understood that if the first confidence is greater than the second confidence, the first container type is determined to be the target container type, otherwise the second container type is determined to be the target container type. If the confidence given by the container classification network model is the highest, the sample container is determined to be the first container type; if the confidence given by the container detection network model is the highest, the sample container is determined to be the second container type. In other words, it is determined whether the two container types are consistent, and if they are consistent, they are of that type; if they are inconsistent, the container type with the highest confidence is selected as the final container type.

[0085] The container classification network model in this application adopts a deep learning convolutional neural network (CNN), which performs well in image classification tasks. First, the constructed convolutional neural network (CNN) is trained through a large number of sample container images so that the CNN model can learn the characteristics of different types of sample containers. Then, in the recognition stage, the sample container image to be identified is input into the trained convolutional neural network (CNN), and the convolutional neural network (CNN) automatically extracts the features in the sample container image and classifies it to obtain the first container type. At the same time, the container detection network model performs segmentation processing on the sample container image, and separately segments the region of interest corresponding to the sample container from the sample container image. The container detection network model automatically extracts the various features of the container in the region of interest, and compares them with the pre-written known different types of sample container features to obtain the second container type. The first container type and the second container type are verified to obtain the target container type, thereby improving the recognition accuracy.

[0086] This application uses two network models to identify the category of sample containers, verifies and optimizes the identification results, and improves the accuracy and stability of identification.

[0087] In one implementation, the method further includes:

[0088] The sample container image is subjected to offset detection and tilt detection, and whether the sample container image is within the detection range is determined based on the detection results, and the image acquisition position or angle adjustment information of the sample container image is output. The position and angle of the camera in the visual module are fixed, and the fixed position and angle can capture the front view of the sample container, which is beneficial to improving the recognition accuracy of the container classification network model and the container detection network model. Due to the vibration of the instrument, the camera may be offset or tilted, resulting in the container in the sample container image captured by the camera not being in the center of the image, or having a certain tilt, resulting in a decrease in recognition accuracy. Therefore, the present application adds offset detection and tilt detection to ensure that the sample container image captured by the camera is within an easily recognizable range.

[0089] Further, performing offset detection and tilt detection on the sample container image, and determining whether the sample container image is within the detection range according to the detection results, includes:

[0090] S511, extract the central axis of the sample container image to obtain the target central axis; compare the target central axis with the set standard central axis; if the horizontal deviation between the target central axis and the standard central axis is within the set threshold range, it is determined that the sample container image is within the horizontal detection range, and the offset detection is determined to be passed. If the horizontal deviation between the target central axis and the standard central axis exceeds the set threshold range, it is determined that the sample container image exceeds the horizontal detection range, the offset detection is determined to be failed, and a horizontal offset prompt message is output according to the horizontal deviation. Since the cylindrical sample container is generally a symmetrical figure, the sample container in the sample container image is segmented and extracted from the background to obtain a sample container object map, and the extracted image containing only the sample container (sample container object map) is treated as a symmetrical figure, and the center of the pixel point of the segmented sample container object map is taken as the target central axis.

[0091] The central axis of an image refers to the center line of an object in the image. In this application, it refers to the center line of the sample container in the vertical direction, which is the center line of the contour line of the sample container. Exemplarily, using Python to extract the central axis in the sample container image includes the following steps:

[0092] import cv2

[0093] import numpy as np

[0094] #Read sample container image

[0095] image=cv2.imread('image.jpg')

[0096] gray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)

[0097] # Binarize the sample container image

[0098] ret,thresh=cv2.threshold(gray,127,255,cv2.THRESH_BINARY)

[0099] #Perform edge detection on the sample container in the sample container image

[0100] edges=cv2.Canny(thresh,100,200)

[0101] #Find the outline of the sample container in the sample container image

[0102] contours,hierarchy=cv2.findContours(edges,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

[0103] #Extract the center line of the sample container

[0104] for contour in contours:

[0105] # Calculate the minimum enclosing rectangle

[0106] rect=cv2.minAreaRect(contour)

[0107] box = cv2.boxPoints(rect)

[0108] box = np.int0(box)

[0109] #Draw the center line

[0110] cv2.drawContours(image,[box],0,0,255,0,2)

[0111] S512, edge detection and line detection are used to calculate the slope of the sample container in the sample container image; if the slope is within the preset slope range, the tilt detection is confirmed to be passed; if the slope exceeds the preset slope range, the tilt detection is confirmed to be failed, and tilt prompt information is output according to the slope. The sample container object in the sample container image is segmented and extracted from the background to obtain a sample container object map. In the sample container object map, the outline of the cylindrical sample container is the edge of the sample container object map, and there are two parallel vertical lines on the left and right in the sample container object map.

[0112] Exemplarily, the Canny edge detection method is used to detect the edge of the sample container in the sample container image; based on the sample container image with the edge of the sample container detected, the Hough transform line detection method is used to detect the straight line; a straight line slope threshold is set, the detected straight lines are sorted according to horizontal straight lines and vertical lines, and two vertical lines of the sample container are screened out, and the slope of the sample container is determined based on the two vertical lines.

[0113] S513: If it is confirmed that the offset detection has passed and the tilt detection has passed, then output prompt information indicating that the image acquisition position and angle are correct; otherwise, output corresponding adjustment information according to the horizontal deviation or the slope.

[0114] Furthermore, the present application also includes correcting the sample container image that exceeds the preset slope range. If the image is detected to be tilted, the image can be corrected by rotating it to restore it to a normal state.

[0115] In one implementation, the first neural network includes a first YOLOv8 network; Figure 4 As shown in Figure 1, the first YOLOv8 network includes the following connected modules: feature extraction backbone module (Backbone) and regression classification module (Head). Backbone uses C2f module and improves feature extraction capability through Bottleneck Block and SPPF modules. Head is responsible for target detection and outputs the recognized sample container category and the first confidence.

[0116] The preset Labelme plug-in is used to annotate the regions of interest in each image included in the first target image data set to obtain a first training data set. Specifically, the Labelme plug-in is used to perform instance segmentation and annotation on the collected sample container images, and the sample container region (region of interest) in the sample container image is annotated according to the json format standard to obtain a json format annotation file; the json format annotation file is converted into a txt format annotation file suitable for YOLOv8 algorithm network structure training using Python code; according to the txt format annotation file, a first annotated training data set is established, in which the annotated region is the sample container region.

[0117] The second neural network includes a second YOLOv8 network. Figure 5As shown, the second YOLOv8 network includes: a feature extraction backbone module (Backbone), a feature fusion module (Neck) and a regression classification module (Head) connected in sequence. Among them, Backbone adopts the C2f module, and the C2f module includes: a 1*1 convolution layer, a feature map segmentation layer, a 3*3 convolution layer, a 3*3 convolution layer, a feature map merging layer and a 1*1 convolution layer connected in sequence, and the feature extraction capability is improved through the Bottleneck Block and SPPF modules. Neck adopts the PAN-FAN structure and performs multi-scale feature fusion through PAN and FAN. Head is responsible for target detection and outputs the predicted bounding box, sample container category and the corresponding second confidence. Similarly, the second target image dataset for training the second YOLOv8 network can be annotated using the Labelme plug-in.

[0118] The specific process of training the second YOLOv8 semantic segmentation network and identifying the container type includes the following steps:

[0119] S611, taking historical sample container images taken of the sample containers on the sample container holder as a training data set, labeling the historical sample container images using Labelme and outputting a mask image set.

[0120] S612, use the PyTorch framework to build the YOLOv8 semantic segmentation network model, and set the network training parameters according to requirements. The network training parameters include model settings, training settings, verification settings, and test settings.

[0121] S613, after the setting is completed, the second YOLOv8 network is loaded and training is started using the GPU. After the training is completed, the YOLOv8 semantic segmentation network with the best performance in the verification process is obtained as the container detection network model (also called the YOLOv8 semantic segmentation network model).

[0122] S614, input the sample container image acquired in real time into the YOLOv8 semantic segmentation network model, and output the mask image information. Specifically, the YOLOv8 semantic segmentation network model is used to perform target detection and semantic segmentation on the sample container image, identify the target area of ​​interest, create a mask contour image (mask image) of the target of interest based on its semantic segmentation result, and extract features from the mask image to obtain container features, which include contour data and image grayscale data.

[0123] S615, matching the container features with the features of each known container type in the preset feature library, specifically including using the SIFT algorithm to calculate the similarity of the sample container image, and taking the type of the image with the highest similarity as the result output.

[0124] Furthermore, in the process of sample tube identification, hospitals use a large number of common scenarios of test tubes and adapters, while other special usage scenarios and unseen scenarios account for a minority. However, these minority scenarios have similar features to common scenarios, which causes the model to tend to identify common scenarios and ignore the identification of other scenarios, resulting in misidentification in minority scenarios. Therefore, the container classification network model and the container detection network model described in this application both use a dynamic weight balance loss function and a Focal Loss loss function during the training process; wherein, the category weight in the dynamic weight balance loss function is determined based on the first parameter in the Focal Loss loss function being less than a preset threshold as the target. This makes the features of special scenarios and unseen minority scenarios more distinguishable. The specific formula of the dynamic weight balance loss function is as follows, where ω i The update method is calculated according to the Focal Loss loss function. When the Focal Loss loss is less than the preset threshold, ω i No longer updated, the overall function can better distinguish between common scenarios and rare scenarios.

[0125] Among them, the formula of the dynamic weight balance loss function is as follows:

[0126]

[0127] Among them, ω i represents the weight assigned to each category (referred to as category weight), and W represents the sum of all weights. This loss function dynamically adjusts the weights during the training process. Preferably, the category weights ω are updated regularly. i To reflect the changes in class distribution during training, N represents the total number of samples, y i represents the value of the i-th category in the true distribution, p i represents the current classification (i-th category) ratio, and L represents the dynamic weight balance loss.

[0128] The Focal Loss loss function formula is as follows:

[0129] FL(p t )=-α t (1-p t ) γ log(p t )

[0130] p t Represents the model's predicted probability for label 1, α t represents the balance factor, α is adjusted according to the category frequency, and γ represents the adjustment factor, which reduces the weight of simple samples and focuses on samples of a few scenes. Preferably, γ = 2. tThe size of represents the degree of difficulty in classifying the sample. When training the model, the model pays more attention to the difficult-to-classify samples. Therefore, it will consider increasing the proportion of difficult-to-classify samples in the Focal Loss function. t From the analysis, we can see that the p of difficult-to-classify samples t Small value, 1-p t Large; easy to classify samples p t Large value, 1-p t The value is small. Regardless of whether it is a difficult or easy sample, Focal Loss has been attenuated relative to the original CEloss, but the attenuation of difficult samples is less than that of easy samples. The hyperparameter γ here determines the degree of attenuation. The larger the γ, the more obvious the loss attenuation. In Focal Loss, compared with difficult samples, easy samples are attenuated more, thereby increasing the weight of difficult samples in disguise. When α is less than the preset threshold, the attenuation change is not obvious, so the impact on the change of classification weight gradually decreases, and the classification weight no longer changes.

[0131] Based on deep learning technology, the YOLOv8 convolutional neural network model for sample container types is built and trained using the collected data; compared with other models, YOLOv8 has several significant advantages in target detection tasks, including its real-time performance, high accuracy, multi-scale prediction capabilities, and adaptive anchor box optimization. YOLOv8 reduces computing resource consumption and improves reasoning speed through model compression and optimization technology.

[0132] Incremental learning and optimization: Incremental update of model parameters to adapt to changing sample container types.

[0133] The present application specifically has the following advantages:

[0134] High-accuracy recognition system: After a large number of experimental verifications, the accuracy of this application in the sample container type recognition task has reached more than 99.7%, which is significantly higher than the recognition level of traditional methods.

[0135] Efficient resource utilization: Through lightweight model design, the inference time is shortened to milliseconds, and it can run efficiently on ordinary GPUs, reducing hardware costs.

[0136] The constructed neural network model automatically learns the container features in the image and efficiently classifies and recognizes these container features. Compared with traditional image processing methods, this application has stronger robustness and generalization ability, and can achieve accurate recognition in complex environments.

[0137] The present application also provides a container type identification device. Exemplarily, the container type identification device includes:

[0138] An image acquisition module, used to acquire an image of a sample container obtained by photographing the sample container on the sample rack;

[0139] A classification module, used for inputting the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence level of the sample container;

[0140] A detection module, used for inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container;

[0141] The decision module is used to determine the target container type from the first container type and the second container type according to the first confidence level and the second confidence level.

[0142] It can be understood that the device of this embodiment corresponds to the container type identification method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0143] The present application also provides an experimental control device. Exemplarily, the experimental control device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the experimental control device to execute the functions of each module in the above-mentioned container type identification method or the above-mentioned container type identification device.

[0144] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0145] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0146] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned experimental control device. For example, the computer-readable storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0147] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0149] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0150] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for identifying container type, characterized in that: include: Acquire a sample container image obtained by photographing the sample container on the sample rack; Inputting the sample container image into a container classification network model for feature recognition to obtain a first container type and a first confidence level of the sample container; Inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container; A target container type is determined from among the first container type and the second container type according to the first confidence level and the second confidence level.

2. The container type identification method according to claim 1, characterized in that: Also includes: Performing offset detection and tilt detection on the sample container image, determining whether the sample container image is within a detection range according to the detection results, and outputting image acquisition position or angle adjustment information of the sample container image.

3. The container type identification method according to claim 2, characterized in that: The performing offset detection and tilt detection on the sample container image and determining whether the sample container image is within a detection range according to the detection results includes: Extracting the central axis of the sample container image to obtain the target central axis; comparing the target central axis with a set standard central axis; if the horizontal deviation between the target central axis and the standard central axis is within a set threshold range, determining that the sample container image is within a horizontal detection range, and determining that the offset detection has passed; Using edge detection and straight line detection to calculate the slope of the sample container in the sample container image; if the slope is within a preset slope range, confirming that the tilt detection has passed; If it is confirmed that the offset detection has passed and the tilt detection has passed, then a prompt message indicating that the image acquisition position and angle are correct is output; otherwise, corresponding adjustment information is output according to the horizontal deviation or the slope.

4. The container type identification method according to claim 1, characterized in that: The determining the target container type from the first container type and the second container type according to the first confidence level and the second confidence level includes: If the first confidence level is greater than the second confidence level, the first container type is determined to be the target container type; otherwise, the second container type is determined to be the target container type.

5. The container type identification method according to claim 1, characterized in that: Inputting the sample container image into a container detection network model for semantic segmentation and feature recognition to obtain a second container type and a second confidence level of the sample container, including: The container detection network model performs semantic segmentation on the sample container image to obtain a target region of interest; and obtains a mask image containing the outline of the sample container according to the target region; Extracting features from the mask image to obtain container features; The container feature is matched with features of each known container type in a preset feature library to obtain the second container type and the second confidence level.

6. The container type identification method according to claim 1, characterized in that: Also includes at least one of the following: Item 1: The container classification network model is trained using the following method: Using a preset plug-in to mark the region of interest in each image included in the first target image data set to obtain a first training data set; Using the first training data set to train the constructed first neural network to obtain the container classification network model; Item 2: The container detection network model is trained using the following method: Using a preset plug-in to mark the regions of interest in each image included in the second target image data set, and outputting a mask image corresponding to each region of interest; The labeled second target image data set and the corresponding mask images are used as the second training data set to train the constructed second neural network to obtain the container detection network model.

7. The container type identification method according to claim 6, characterized in that: The first neural network includes a first YOLOv8 network; the first YOLOv8 network includes a feature extraction backbone module and a regression classification module connected in sequence; The second neural network includes a second YOLOv8 network; the second YOLOv8 network includes a feature extraction backbone module, a feature fusion module and a regression classification module connected in sequence.

8. The container type identification method according to any one of claims 1 to 7, characterized in that: The container classification network model and the container detection network model both adopt a dynamic weight balancing loss function and a FocalLoss loss function during training; wherein the category weight in the dynamic weight balancing loss function is determined based on the first parameter in the Focal Loss loss function being less than a preset threshold as the target.

9. An experimental control device, characterized in that: The experimental control device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the container type identification method according to any one of claims 1 to 8.

10. An experimental system, characterized in that: The experimental system includes: a camera and an experimental control device; The camera is used to take a picture of the sample container on the sample rack to obtain a sample container image, and the experiment control device is used to perform the container type recognition method described in any one of claims 1 to 8 on the sample container image.

Images