Container lock lever bending measurement and classification method and device

Abstract

The application provides a container lock rod curvature measurement and classification method and device, which is applied to the technical field of container damage detection, accurately detects and measures the curvature of the lock rod in the container door image through a trained instance segmentation model, and classifies the curvature of the lock rod according to a preset curvature threshold interval, thereby significantly improving the accuracy and reliability of the lock rod curvature measurement, effectively avoiding the inaccurate measurement problem caused by the diverse lock rod bending form and background interference, labeling the lock rod profile and the corner profile during the model training process, calculating the initial curvature value of the lock rod, correcting the initial curvature value through perspective transformation to obtain the corrected lock rod curvature value, and using the corrected curvature value as an additional label to construct a data set for training the instance segmentation model, thereby significantly reducing the dependence on a large number of lock rod bending picture labeling data and reducing the data collection and labeling cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of container damage detection, in particular to a container lock rod bending degree measurement and classification method and device suitable for container application scenarios. BACKGROUND

[0002] During long-distance transportation and frequent loading and unloading of containers, the lock rods of container doors often suffer damage due to external impact, improper loading and unloading, etc., resulting in different degrees of bending deformation of the lock rods, and even breakage or loss in severe cases. These damages not only affect the normal use of the container, but also may pose a threat to the safety of the goods. Therefore, accurately identifying the bending degree of the lock rod is crucial for the maintenance and safe use of the container.

[0003] Currently, the detection of the bending degree of the container lock rod mainly relies on manual observation and measurement, which is low in efficiency and easily affected by human factors. In recent years, some regions have attempted to use target detection technology to detect the bending of the lock rod, but the detection accuracy is very low and automatic detection and measurement of the bending of the lock rod cannot be achieved.

[0004] Existing automatic detection methods for lock rod bending rely on a large amount of lock rod bending picture labeling data. However, the shape of the lock rod bending is diverse and has no fixed rules, which makes the data collection and labeling process costly and time-consuming. Some solutions can directly predict the category and location information of the container damage from the image, but in the case of a lack of lock rod bending samples, a large amount of cost and time is needed to collect and synthesize training data, such as invention patents CN112819793, CN115187535 and CN115222697. Some solutions can solve the problem of occasional damage sample scarcity and damage severity identification, but when encountering unobserved bending shapes, the detection success rate is extremely low, such as invention patent CN117291921A. In addition, existing solutions cannot further distinguish the bending degree of the lock rod.

[0005] Moreover, traditional image segmentation algorithms usually rely on image gray values to achieve image segmentation, such as edge detection segmentation algorithms. However, these traditional segmentation algorithms often cannot effectively distinguish between the background and the lock rod area when faced with complex backgrounds and other components such as doors, especially when the lock rod is severely bent, causing the lock rod image to overlap, making it impossible to accurately segment the lock rod area, thus making the identification of the lock rod bending unreliable.

[0006] Therefore, there is a need for a new container lock rod bending degree measurement and classification scheme. SUMMARY

[0007] In view of this, the embodiments of this specification provide a method and device for measuring and classifying the curvature of container lock rods. By segmenting the container door image, the lock rod contour is extracted to calculate the initial curvature value of the lock rod, and the corner piece contour is extracted to calculate the scale correction factor. The scale correction factor is then used to correct the initial curvature value of the lock rod. Finally, the degree of curvature of the lock rod is classified according to a preset curvature threshold range, thereby capturing the geometric morphological information of the lock rod and obtaining the curvature of the segmented lock rod instance.

[0008] The embodiments of this specification provide the following technical solutions:

[0009] The embodiments of this specification provide a method for measuring and classifying the curvature of a container lock rod, including:

[0010] Input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected curvature value of the lock rod instance;

[0011] Classifying the corrected curvature values ​​according to a preset curvature threshold range to obtain a curvature grade corresponding to each locking rod instance;

[0012] The instance segmentation model is obtained by the following training method:

[0013] Mark the lock rod contour and corner piece contour in the container door image to obtain marked contour data;

[0014] Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance;

[0015] Extract the corner points of the corner pieces and perform perspective transformation to obtain the scale correction factor;

[0016] Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value;

[0017] Using the corrected locking rod curvature value as an additional label to construct a data set;

[0018] An instance segmentation model is trained using the dataset.

[0019] The embodiments of this specification also provide a container lock rod curvature measurement and classification device, comprising:

[0020] The curvature value measurement module is used to input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected curvature value of the lock rod instance;

[0021] The instance segmentation model is obtained by the following training method:

[0022] Mark the lock rod contour and corner piece contour in the container door image to obtain marked contour data;

[0023] Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance;

[0024] Extract the corner points of the corner pieces and perform perspective transformation to obtain the scale correction factor;

[0025] Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value;

[0026] Using the corrected locking rod curvature value as an additional label to construct a data set;

[0027] Training an instance segmentation model using the dataset;

[0028] The curvature classification module is used to classify the corrected curvature value according to a preset curvature threshold range to obtain a curvature grade corresponding to each locking rod instance.

[0029] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0030] On the one hand, the lock rod contour is accurately extracted through the trained instance segmentation model, and the initial curvature value of the lock rod is corrected by combining the scale correction factor calculated by the corner piece contour. This can effectively correct the shape deformation of the lock rod caused by changes in shooting angle and viewing angle, thereby significantly improving the accuracy and reliability of the lock rod curvature measurement, and effectively avoiding the measurement inaccuracy caused by the diverse bending shapes of the lock rod and background interference.

[0031] Secondly, during the model training process, a data set containing the corrected lock rod curvature values ​​is constructed for training, so that the model can learn the geometric characteristics of the lock rod curvature, which not only enhances the model's ability to recognize different forms of lock rod curvatures, but also improves the model's generalization ability.

[0032] Thirdly, only contour annotation is required, and the corrected lock rod curvature value is included in the data set as an additional label, which significantly reduces the dependence on a large amount of lock rod bending image annotation data, significantly reduces data collection and annotation costs, and is very easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 This is a process of measuring and classifying the curvature of container lock rods in this application Figure 1 ;

[0035] Figure 2 This is a process of measuring and classifying the curvature of container lock rods in this application Figure 2 ;

[0036] Figure 3 This is a diagram of a container door lock rod in this application;

[0037] Figure 4 This is a skeleton diagram of a container door lock rod in this application;

[0038] Figure 5 This is a curve fitting diagram of a container door lock rod in this application. DETAILED DESCRIPTION

[0039] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0040] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the features in the following embodiments and embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of this application.

[0041] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspect described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0042] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. The illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0043] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples, however, one skilled in the art will appreciate that the examples can be practiced without these specific details.

[0044] The lock rods of container doors are often damaged by external impacts, improper loading and unloading, and other factors, causing the lock rods to bend and deform to varying degrees. In severe cases, they may even break or become missing. The bending of the lock rods varies greatly and there is no fixed rule. Currently, the existing methods for detecting lock rod bending mainly include the following:

[0045] First, it relies on manual observation and measurement, which is inefficient and easily affected by human factors, resulting in insufficient accuracy and reliability of measurement results.

[0046] Secondly, methods such as those described in invention patents CN112819793, CN115187535, and CN115222697, while able to directly predict the type and location of container damage from images, rely on a large amount of annotated data from images of bent lock rods. However, bent lock rods are sporadic events, and real samples are scarce. In order to meet the training model requirements, it takes a lot of cost and time to collect synthetic training data. Even though invention patent CN117291921A can address the scarcity of sporadic damage samples and the problem of identifying damage severity, the model has an extremely low success rate for detecting bent lock rods with diverse and irregular shapes when encountering unseen bent lock rods, and is unable to further distinguish the degree of curvature of the lock rods.

[0047] Third, traditional image segmentation methods, such as edge detection segmentation algorithms, rely on the grayscale value of the image to achieve image segmentation, which makes it difficult to effectively distinguish the interference of the background and other parts of the door. Especially when the lock rod is severely bent and the lock rod image overlaps, it is impossible to accurately segment the lock rod area, resulting in unreliable recognition of the bending of the lock rod.

[0048] In view of this, the inventors discovered through research and improvement exploration that the bending shapes of the lock rod are diverse and have no fixed rules, resulting in a scarcity of real samples, which in turn makes data collection and labeling costly and time-consuming. Moreover, the existing detection scheme can only determine whether the lock rod is bent, but cannot quantify the degree of bending of the lock rod. In addition, the traditional image segmentation algorithm cannot accurately identify the lock rod area in a complex background, which seriously affects the efficiency and accuracy of the lock rod bending detection, and restricts the safety and maintenance of container transportation and loading and unloading.

[0049] Based on this, the embodiment of this specification proposes a method for measuring and classifying the curvature of container lock rods: the overall idea is: through a trained instance segmentation model, the lock rods in the container door image are accurately detected and the curvature is measured, and the curvature degree of the lock rods is classified according to the preset curvature threshold range. During the model training process, the lock rod contour and the corner piece contour are marked, the initial curvature value of the lock rod is calculated, the scale correction factor is calculated through perspective transformation, the initial curvature value is geometrically corrected, and the corrected lock rod curvature value is obtained. These corrected curvature values ​​are used as additional labels to construct a data set for training the instance segmentation model.

[0050] The following describes the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0051] like Figure 1 As shown, the embodiment of this specification provides a method for measuring and classifying the curvature of a container lock rod, comprising:

[0052] Step S1: Input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected lock rod curvature value;

[0053] Step S2: classifying the curvature values ​​of the locking rods according to a preset curvature threshold range to obtain a curvature grade corresponding to each locking rod;

[0054] The instance segmentation model is obtained by the following training method:

[0055] Mark the lock rod contour and corner piece contour in the container door image to obtain marked contour data;

[0056] Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance;

[0057] Extract the corner points of the corner pieces and perform perspective transformation to obtain the scale correction factor;

[0058] Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value;

[0059] Using the corrected locking rod curvature value as an additional label to construct a data set;

[0060] An instance segmentation model is trained using the dataset.

[0061] In the implementation, untrained container door angle images are used as test images to input into the trained instance segmentation model, and the corrected curvature value of each lock rod instance is output.

[0062] It should be noted that the instance segmentation model can accurately extract the lock rod contour and calculate the curvature of the lock rod, which is then corrected using a scale correction factor and finally outputs the corrected curvature value, ensuring the accuracy of the detection results.

[0063] Then, based on the preset curvature threshold range, the degree of bending of the locking rod can be divided into different levels, such as no bending, slight bending, moderate bending and severe bending, etc., providing a basis for subsequent maintenance and safety assessment.

[0064] When training the instance segmentation model, polygon annotation can be used to obtain image data of the container door lock rod outline and the complete corner piece outline. Then, based on the annotated outline data, corner perspective transformation is performed to calculate the initial curvature value and scale correction factor of each lock rod. The scale correction factor is then used to perform geometric correction on the initial curvature value to obtain the corrected curvature and add it to the dataset as additional annotation information, thereby reducing dependence on a large amount of annotated data, reducing the cost and time of data collection and annotation, and improving the efficiency of model development.

[0065] This application can randomly divide the data set containing additional annotation information into training set, test set, and validation set for subsequent model training and validation, significantly improving the accuracy and reliability of lock rod curvature measurement.

[0066] In some embodiments, obtaining an initial curvature value of each locking rod instance based on the annotated contour data includes:

[0067] Perform B-spline curve fitting on the lock rod contour points to obtain a fitting curve;

[0068] Based on the fitting curve, an initial curvature value is obtained.

[0069] like Figure 3 As shown, the locking rod exhibits different degrees of bending, which makes the shape of the locking rod complex. If the contour points are directly used to calculate the curvature, the result may be affected by noise and shape irregularities, thereby reducing the accuracy of the measurement.

[0070] like Figure 4As shown, the skeleton of the lock rod can be extracted from the original drawing first, the shape of the lock rod is simplified, and the main structural features of the lock rod are retained, then the B-spline curve fitting technology is used to fit the contour points of the lock rod in the skeleton, a smooth curve is obtained, see Figure 5 , which can more accurately describe the actual shape of the lock rod, not only reduces the influence of noise, but also improves the reliability of curvature calculation, so as to more accurately evaluate the bending degree of the lock rod and effectively quantify the bending degree of the lock rod, thereby providing important data support for the safe use and maintenance of the container.

[0071] Let the direct coordinate equation of the curve be: ;

[0072] Let the parametric equation of the curve be:

[0073] ;

[0074] The initial curvature value of the lock rod contour is obtained, which is expressed by the formula:

[0075] ; wherein, t is the parameter.

[0076] In some embodiments, the geometric correction comprises:

[0077] extracting four corner points of the corner piece contour, establishing a perspective mapping relationship between the corner points and the corresponding actual size coordinates, and obtaining the corrected lock rod contour;

[0078] obtaining a scale correction factor according to the minimum circumscribed rectangle size of the corner piece contour and the standard corner piece actual size;

[0079] correcting the initial curvature value by using the scale correction factor to obtain the corrected lock rod curvature value.

[0080] In the implementation, four corner point coordinates of the corner piece are labeled and extracted from the container door image , the perspective mapping relationship between the corner points in the image coordinate system and the actual size coordinate system is established by using the four corner points and the corner point coordinates of the standard corner piece , the lock rod contour in the image is corrected to the actual size, so as to obtain the corrected lock rod contour, thereby eliminating the deformation of the lock rod contour caused by the shooting angle.

[0081] Then, the actual size of the standard corner piece is divided by the minimum circumscribed rectangle size of the image corner piece contour to obtain the scale correction factor , and the correction formula is: .

[0082] The corrected lock rod curvature value is obtained .

[0083] In some embodiments, when training the instance segmentation model, a total loss function is defined, where the total loss function includes target detection loss, segmentation loss, and curvature loss; wherein the expression of the total loss function is:

[0084] ;in, represents the target detection loss; represents the segmentation loss; represents curvature loss; is the weight coefficient, and .

[0085] During the model training process, this application uses the curvature calculation and correction module to obtain the curvature loss function, and incorporates it into the weighted target prediction and segmentation losses to obtain the total model loss function, thereby more accurately capturing the geometric morphological information of the lock rod and obtaining the curvature of the segmented lock rod instance.

[0086] In some embodiments, when training the instance segmentation model, the loss function weight and the learning rate are adjusted according to the curvature loss ratio of the instance segmentation model, thereby improving the curvature prediction accuracy.

[0087] During implementation, an independent validation set was used to evaluate the performance of the model, monitor the segmentation accuracy and curvature prediction error, and adjust hyperparameters such as the loss function weight and learning rate based on the validation results to optimize the model performance.

[0088] In some embodiments, the container lock bar curvature measurement and classification method further includes:

[0089] According to a preset confidence threshold, lock bar instances whose confidences are higher than the preset confidence threshold are screened out.

[0090] In implementation, untrained images of container door angles are fed into an instance segmentation model (i.e., a container door lock rod instance segmentation model) as the test images. The model outputs the instance category, bounding box, mask, confidence score, and corrected curvature. The predicted masks are filtered to retain lock rod instances with high confidence.

[0091] In some embodiments, the preset curvature threshold interval is set according to the corresponding relationship between the curvature of the locking rod and the severity.

[0092] Specifically, after obtaining the curvature value, a corresponding relationship between the curvature value of the lock rod and the severity of the curvature is established according to historical statistical data of the curvature and severity of the lock rod, and a threshold interval is set to divide the curvature value into different curvature levels.

[0093] In some embodiments, the instance segmentation model is constructed based on deep learning, and the instance segmentation model includes: a backbone network, a feature module, a detection branch, a segmentation branch, and a curvature calculation and correction module;

[0094] The backbone network is used to extract image features;

[0095] The feature module is used to fuse lock rod instances of different sizes;

[0096] The detection branch is used to predict the instance category, the locking rod instance position and the locking rod instance mask confidence;

[0097] The segmentation branch is used to generate a lock rod instance segmentation mask and mask coefficients;

[0098] The curvature calculation and correction module is used to calculate the curvature value of the locking rod and perform correction to obtain the final curvature value of the locking rod.

[0099] Specifically, the backbone network can choose ResNet-50, improve the feature extraction capability through the pre-training model, and fuse feature maps of different scales through the feature fusion (Feature Pyramid Network, FPN) structure. In the prediction head (Prediction Head) structure, based on the feature maps of different scales, a large number of anchor boxes of different sizes and aspect ratios are generated, and on this basis, the target category, position and mask confidence (Mask Coefficients) are predicted. Using several layers of convolution operations, based on the feature layer P3 with the largest space, the ProtoNet branch generates a fixed-dimensional mask prototype.

[0100] The data of the prediction branch and the segmentation branch are linearly combined to obtain the final instance segmentation mask, and then the resulting data is post-processed, mainly including cropping the mask prototype and binarizing the pixel values ​​in the mask prototype to ensure the generation of an accurate instance segmentation mask.

[0101] The lock rod's contour points are extracted, and a perspective transformation is performed on the corner points. Then, a B-spline curve is fitted to the contour points to create a smooth curve. The curvature value is calculated based on the fitted curve and finally corrected to measure the lock rod's curvature.

[0102] In some embodiments, the detection branch further utilizes fast non-maximum suppression to eliminate overlapping target areas.

[0103] Specifically, for the large number of region of interest boxes generated by the prediction head, fast non-maximum suppression (FastNMS) is used to eliminate overlapping target areas and retain the best detection results.

[0104] In some embodiments, the curvature calculation and correction module includes: a curvature calculation branch and a curvature correction branch;

[0105] The curvature calculation branch is used to extract the contour points of the lock rod from the lock rod instance segmentation mask, and use a B-spline curve to fit the extracted contour points to obtain a lock rod contour; based on the lock rod contour, obtain a curvature value of the lock rod;

[0106] The curvature correction branch is used to correct the curvature value according to the correction factor obtained by perspective transformation to obtain the final curvature value of the locking rod.

[0107] During implementation, the lock rod profile is obtained according to the curvature calculation branch, the curvature correction branch performs perspective transformation on the lock rod profile, and then obtains a scale correction factor to correct the calculated curvature.

[0108] Here is another example, which is a schematic illustration formed by combining the above examples.

[0109] The present invention provides a method for automatically measuring and classifying the curvature of container lock bars based on deep learning, which may include the following steps:

[0110] Building instance segmentation model architecture based on deep learning:

[0111] The instance segmentation model architecture includes a backbone network, a feature module, a detection branch, a segmentation branch, and a curvature calculation and correction module.

[0112] in,

[0113] Detection branch design: predict the instance category in the image, predict the specific position and size of the lock rod instance in the image, and predict the confidence of the mask of each lock rod instance.

[0114] Segmentation branch design: Generate a fixed number of prototype masks and generate mask coefficients for each instance.

[0115] Curvature calculation branch design: extract the contour points of the locking rod instance, fit the contour points based on the B-spline curve to obtain a smooth curve, and calculate the curvature value based on the fitting curve and scale correction factor.

[0116] Curvature correction branch design: The four corner points of the corner piece are extracted and used to perform perspective transformation to correct the lock rod outline. Then, the scale correction factor is calculated based on the minimum circumscribed rectangle size of the corner piece outline and the actual size of the standard corner piece.

[0117] Curvature calculation and correction module design: It includes a curvature calculation branch and a curvature correction branch. The lock rod profile is obtained according to the curvature calculation branch. The curvature correction branch performs perspective transformation on the lock rod profile, and then obtains the scale correction factor to correct the calculated curvature.

[0118] The specific process of overall network model design is as follows:

[0119] (1) Select ResNet-50 as the backbone network and improve the feature extraction capability through pre-training model.

[0120] (2) Feature maps of different scales are fused through the FPN (Feature Pyramid Network) structure.

[0121] (3) In the prediction head structure, a large number of anchor boxes of different sizes and aspect ratios are generated based on feature maps of different scales, and the target category, position and mask confidence (Mask Coefficients) are predicted on this basis.

[0122] (4) Using several layers of convolution operations, the ProtoNet branch generates a fixed-dimensional mask prototype based on the spatially largest feature layer P3.

[0123] (5) For the large number of region of interest boxes generated by the prediction head, fast non-maximum suppression (FastNMS) is used to eliminate overlapping target areas.

[0124] (6) The data of the prediction branch and the segmentation branch are linearly combined to obtain the final instance segmentation mask, and then the resulting data is post-processed, mainly including cropping the mask prototype and binarizing the pixel values ​​in the mask prototype to ensure the generation of an accurate instance segmentation mask.

[0125] (7) The contour points of the lock rod instance are extracted, and the corner points are used for perspective transformation. Then, a smooth curve is obtained by fitting the contour points based on a B-spline curve. The curvature value is calculated based on the fitted curve, and finally, after correction, it is used as a measure of the curvature of the lock rod.

[0126] Dataset annotation and preparation:

[0127] Polygon annotation was used to obtain image data of the container door lock rod and complete corner fitting outlines. Based on the annotated outline data, a corner perspective transformation was performed to calculate the curvature of each lock rod. A scale estimation factor was derived using the minimum circumscribed rectangle of the corner fitting and the actual dimensions of the standard corner fitting. The corrected curvature was then added to the dataset as additional annotation information. The dataset was randomly divided into training, testing, and validation sets for subsequent model training and validation.

[0128] The calculation formula can refer to the above example and will not be expanded here.

[0129] Training and evaluating the model:

[0130] An initial instance segmentation model was trained using a dataset of container door lock bar outline images and curvature. This model was then developed for instance segmentation and curvature of container door lock bars. An independent validation set was used to evaluate the model's performance, monitoring segmentation accuracy and curvature prediction error. Based on the validation results, hyperparameters such as the loss function weight and learning rate were adjusted to optimize model performance.

[0131] Use the trained model for measurement and classification:

[0132] like Figure 2 As shown in the figure, the image of the container door to be tested is input into the trained instance segmentation model. First, the image features are extracted through the backbone network ResNet-50 in the model. Then, multi-scale feature fusion is performed using the Feature Pyramid Network (FPN) to enhance the model's ability to detect objects of different sizes. The model is then divided into two branches for processing. The detection branch is responsible for predicting the category, bounding box, and mask confidence, while the segmentation branch generates the prototype mask and mask coefficients through ProtoNet.

[0133] After that, Fast Non-Maximum Suppression (Fast NMS) is applied to remove overlapping detection boxes and retain the optimal detection results. The mask prototype and coefficients are then linearly combined to generate a preliminary segmentation mask. Post-processing operations such as cropping and binarization are then performed on the segmentation results to retain high-confidence lock rod instances.

[0134] On this basis, the lock rod contour points are extracted from the segmentation results, and a smooth curve is obtained by B-spline curve fitting. The initial curvature value is calculated based on the curve parameter equation. At the same time, the lock rod contour is corrected by extracting the corner points of the corner fittings and performing perspective transformation. The scale correction factor is calculated based on the minimum circumscribed rectangle size of the corner fittings and the actual size of the standard corner fittings. The initial curvature value is then geometrically corrected using the scale correction factor to obtain the corrected lock rod curvature value. Finally, the corrected curvature values ​​are classified according to the preset curvature threshold range to determine the curvature grade of each lock rod instance. The final detection results including category, bounding box, mask, curvature and curvature grade are output, realizing the automatic measurement and classification of container lock rod curvature, providing an important basis for the safe use and maintenance of containers.

[0135] This application establishes a one-stage instance segmentation and curvature calculation model, so that the model can not only directly learn the curvature characteristics of the lock rod, but also improve the model's ability to recognize and segment the bent lock rod, so that it can accurately and quickly segment the lock rod area, ensure the feasibility of curvature calculation, thereby improving the accuracy and reliability of automatic detection of lock rod bending, and effectively reducing the difficulty of training, collecting and labeling data and the cost of model development.

[0136] The present invention realizes automatic identification of container lock bars and measurement of curvature only through two-dimensional images and image segmentation technology, provides comprehensive lock bar bending assessment and classification results, avoids the installation of expensive equipment and complex measurement and data processing processes, simplifies the implementation of the damage system, and reduces the hardware cost, maintenance difficulty and overall complexity of the system.

[0137] Based on the same inventive concept, the present application also provides a container lock rod curvature measurement and classification device, comprising:

[0138] The curvature value measurement module is used to input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected curvature value of the lock rod instance;

[0139] The instance segmentation model is obtained by the following training method:

[0140] Mark the lock rod contour and corner piece contour in the container door image to obtain marked contour data;

[0141] Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance;

[0142] Extract the corner points of the corner pieces and perform perspective transformation to obtain the scale correction factor;

[0143] Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value;

[0144] Using the corrected locking rod curvature value as an additional label to construct a data set;

[0145] Training an instance segmentation model using the dataset;

[0146] The curvature classification module is used to classify the corrected curvature value according to a preset curvature threshold range to obtain a curvature grade corresponding to each locking rod instance.

[0147] In implementation, the instance segmentation model can be trained externally, and the trained model file can be directly deployed to the measurement and classification device.

[0148] In some embodiments, the container lock bar curvature measurement and classification device further includes:

[0149] Instance segmentation model training module: used to annotate the lock rod and corner piece contours in container door images to obtain annotated contour data;

[0150] Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance;

[0151] Extract the corner points of the corner pieces and perform perspective transformation to obtain the scale correction factor;

[0152] Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value;

[0153] Using the corrected locking rod curvature value as an additional label to construct a data set;

[0154] An instance segmentation model is trained using the dataset.

[0155] In this specification, the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments described later, the description is relatively simple, and the relevant parts can be referred to the partial description of the previous embodiments.

[0156] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for measuring and classifying the curvature of container lock rods, characterized in that: include: Input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected lock rod curvature value; Classifying the curvature values ​​of the locking rods according to a preset curvature threshold range to obtain a curvature grade corresponding to each locking rod; The instance segmentation model is obtained by the following training method: Mark the lock rod contour and corner piece contour in the container door image to obtain marked contour data; Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance; Extract the corner points of the corner piece and perform perspective transformation to obtain the corrected lock rod outline; Obtaining a scale correction factor according to the minimum circumscribed rectangle size of the corner piece outline and the actual size of the standard corner piece; Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value; Using the corrected locking rod curvature value as an additional label to construct a data set; An instance segmentation model is trained using the dataset.

2. The method for measuring and classifying the curvature of container lock bars according to claim 1, characterized in that: The obtaining of an initial curvature value of each locking rod instance based on the annotated contour data includes: Perform B-spline curve fitting on the lock rod contour points to obtain a fitting curve; Based on the fitting curve, an initial curvature value is obtained.

3. The method for measuring and classifying the curvature of container lock bars according to claim 1, characterized in that: The geometric correction includes: Extracting four corner points of the corner piece outline, establishing a perspective mapping relationship between the corner points and corresponding actual size coordinates, and obtaining a corrected lock rod outline; Obtaining a scale correction factor according to the minimum circumscribed rectangle size of the corner piece outline and the actual size of the standard corner piece; The initial curvature value is corrected using the scale correction factor to obtain a corrected locking rod curvature value.

4. The method for measuring and classifying the curvature of container lock bars according to claim 1, characterized in that: When training the instance segmentation model, a total loss function is defined, where the total loss function includes target detection loss, segmentation loss, and curvature loss; wherein the expression of the total loss function is: ;in, represents the target detection loss; represents the segmentation loss; Indicates curvature loss; is the weight coefficient, and .

5. The method for measuring and classifying the curvature of container lock bars according to claim 4, characterized in that: When training the instance segmentation model, the loss function weight and the learning rate are adjusted according to the curvature loss ratio of the instance segmentation model.

6. The method for measuring and classifying the curvature of container lock bars according to claim 1, characterized in that: The container lock rod curvature measurement and classification method further includes: According to a preset confidence threshold, lock bar instances whose confidences are higher than the preset confidence threshold are screened out.

7. The method for measuring and classifying the curvature of container lock bars according to claim 1, characterized in that: The preset curvature threshold interval is set according to the corresponding relationship between the curvature of the locking rod and the severity.

8. The method for measuring and classifying the curvature of container lock bars according to any one of claims 1 to 7, characterized in that: Constructing the instance segmentation model based on deep learning, the instance segmentation model includes: a backbone network, a feature module, a detection branch, a segmentation branch and a curvature calculation and correction module; The backbone network is used to extract image features; The feature module is used to fuse lock rod instances of different sizes; The detection branch is used to predict the instance category, the locking rod instance position and the locking rod instance mask confidence; The segmentation branch is used to generate a lock rod instance segmentation mask and mask coefficients; The curvature calculation and correction module is used to calculate the curvature value of the locking rod and perform correction to obtain the final curvature value of the locking rod.

9. The method for measuring and classifying the curvature of container lock bars according to claim 8, characterized in that: The detection branch also uses fast non-maximum suppression to eliminate overlapping target areas.

10. The method for measuring and classifying the curvature of container lock bars according to claim 8, characterized in that: The curvature calculation and correction module includes: a curvature calculation branch and a curvature correction branch; The curvature calculation branch is used to extract the contour points of the lock rod from the lock rod instance segmentation mask, and use a B-spline curve to fit the extracted contour points to obtain a lock rod contour; based on the lock rod contour, obtain a curvature value of the lock rod; The curvature correction branch is used to correct the curvature value according to the correction factor obtained by perspective transformation to obtain the final curvature value of the locking rod.

11. A container lock rod curvature measurement and classification device, characterized in that: include: The curvature value measurement module is used to input the image of the container door to be tested into the trained instance segmentation model to obtain the corrected curvature value of the lock rod instance; a curvature classification module, configured to classify the corrected curvature values ​​according to a preset curvature threshold interval to obtain a curvature grade corresponding to each locking rod instance; Instance segmentation model training module: used to annotate the lock rod and corner piece contours in container door images to obtain annotated contour data; Based on the annotated contour data, obtaining an initial curvature value of each locking rod instance; Extract the corner points of the corner piece and perform perspective transformation to obtain the corrected lock rod outline; Obtaining a scale correction factor according to the minimum circumscribed rectangle size of the corner piece outline and the actual size of the standard corner piece; Performing geometric correction on the initial curvature value using the scale correction factor to obtain a corrected locking rod curvature value; Using the corrected locking rod curvature value as an additional label to construct a data set; An instance segmentation model is trained using the dataset.

Images