Profile automatic detection method, device and equipment and storage medium

Abstract

The invention relates to a profile automatic detection method and device, equipment and a storage medium. The method comprises the following steps: performing carriage detection on a loading profile image, determining a carriage area image, inputting the carriage area image into a profile detection model for profile detection, and outputting a plurality of profile prediction frames; based on the morphological characteristics of the prediction frames, performing primary screening processing on the plurality of profile prediction frames to obtain a first screening prediction frame set, based on the position characteristics of the prediction frames, performing secondary screening processing on the first screening prediction frame set to obtain a second screening prediction frame set, and based on the distance characteristics of the prediction frames, performing secondary screening processing on the second screening prediction frame set to obtain a second screening prediction frame set; performing third screening processing on the second screening prediction frame set to obtain a third screening prediction frame set; compared with the prior art, the technical scheme of the invention has the advantages that the dependence on computing resources can be reduced, and efficient and accurate automatic detection and counting of the profiles can be realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an automatic profile inspection method, apparatus, equipment and storage medium. Background Technology

[0002] In industrial profile loading scenarios, to improve the efficiency of profile detection and counting, existing methods rely on digital image processing technology or SAM-based image segmentation methods.

[0003] Digital image processing technology enhances profile image features through preprocessing operations such as denoising, contrast enhancement, grayscale conversion, and binarization, and then extracts features such as edges, shapes, colors, and textures. Based on predefined features, profile recognition and counting are achieved. Meanwhile, the SAM-based image segmentation method uses powerful instance segmentation capabilities to segment profiles in dense loading scenarios, and then completes counting statistics based on the segmented instances.

[0004] However, traditional digital image processing techniques have significant shortcomings: on the one hand, they rely heavily on manually extracted profile features and are easily affected by changes in the surrounding environment, slight tilt, and occlusion, resulting in poor robustness; on the other hand, they are weak in detecting small targets in dense scenes and cannot meet the profile counting requirements of loading scenarios; at the same time, although image segmentation methods based on SAM can effectively deal with target detection in dense scenes, they have two obvious drawbacks: first, it is difficult to avoid the influence of interference outside the carriage; second, they have high requirements for computing resources and are time-consuming. Summary of the Invention

[0005] This application provides an automatic profile detection method, apparatus, equipment, and storage medium, which can reduce the dependence on computing resources and achieve efficient and accurate automatic profile detection and counting.

[0006] In a first aspect, this application provides an automatic profile detection method, comprising: performing vehicle compartment detection on a vehicle-mounted profile image to determine a vehicle compartment area image; inputting the vehicle compartment area image into a pre-trained profile detection model, so that the profile detection model performs profile detection on the vehicle compartment area image and outputs multiple profile prediction boxes; performing a first filtering process on the multiple profile prediction boxes based on the prediction box morphology features to obtain a first filtered prediction box set; performing a second filtering process on the first filtered prediction box set based on the prediction box position features to obtain a second filtered prediction box set; performing a third filtering process on the second filtered prediction box set based on the prediction box distance features to obtain a third filtered prediction box set; and performing a counting process on the third filtered prediction box set to obtain the number of profiles in the aluminum material image.

[0007] In one possible implementation, based on the shape features of the prediction frames, a filtering process is performed on the plurality of profile prediction frames to obtain a first set of filtered prediction frames. Specifically, this includes: obtaining the shape features of each of the plurality of profile prediction frames, wherein the shape features include prediction frame length, prediction frame width, prediction frame area, and prediction frame aspect ratio; determining whether the prediction frame length is within a preset length range, determining whether the prediction frame width is within a preset width range, determining whether the prediction frame area is within a preset area range, and determining whether the prediction frame aspect ratio is within a preset aspect ratio range; if any prediction frame length is not within the preset length range, the prediction frame... If the width is not within the preset width range, the predicted frame area is not within the preset area range, or the predicted frame aspect ratio is not within the preset aspect ratio range, then the profile predicted frame corresponding to the current predicted frame shape feature is removed. Based on the predicted frame shape feature, the mean of the predicted frame shape feature and the standard deviation of the predicted frame shape feature are calculated, and a normal distribution confidence interval is constructed based on the mean of the predicted frame shape feature and the standard deviation of the predicted frame shape feature. It is determined whether the current predicted frame shape feature exceeds the normal distribution confidence interval. If so, the profile predicted frame corresponding to the current predicted frame shape feature is removed. All the remaining profile predicted frames are integrated to obtain the first set of filtered predicted frames.

[0008] In one possible implementation, based on the location features of the prediction boxes, a second filtering process is performed on the first set of filtered prediction boxes to obtain a second set of filtered prediction boxes. Specifically, this includes: obtaining the coordinates of the first filtered prediction box corresponding to each first filtered prediction box in the first set of filtered prediction boxes; comparing the coordinates of the first filtered prediction box with the boundary of the rectangle corresponding to the image of the carriage area; if the first filtered prediction box is outside the boundary of the rectangle, then the current first filtered prediction box is removed; if the first filtered prediction box is inside the boundary of the rectangle, then the current first filtered prediction box is retained; and obtaining the coordinates of the first filtered prediction box corresponding to each first filtered prediction box in the first set of filtered prediction boxes. Based on a first ordinate, each first filtering prediction box is sorted in descending order and layered to assign each first filtering prediction box to a matching first filtering prediction box layer. Each first filtering prediction box in the set is traversed layer by layer from bottom to top to obtain multiple layers of prediction box counts. The count of each layer of prediction boxes is compared with a preset layer prediction box count threshold. If the count of a layer of prediction boxes is less than the preset layer prediction box count threshold, all first filtering prediction boxes in the first filtering prediction box layer corresponding to that count are removed. All the remaining first filtering prediction boxes are then integrated to obtain a second filtering prediction box set.

[0009] In one possible implementation, based on the distance feature of the predicted boxes, the second set of filtered predicted boxes is subjected to three filtering processes to obtain a third set of filtered predicted boxes. Specifically, this includes: using a density clustering algorithm based on rectangular distance to cluster the second set of filtered predicted boxes into multiple clusters; determining the largest cluster among the multiple clusters, removing all second filtered predicted boxes corresponding to non-largest clusters among the multiple clusters, and retaining all second filtered predicted boxes corresponding to the largest cluster; and integrating all the retained second filtered predicted boxes to obtain the third set of filtered predicted boxes.

[0010] In one possible implementation, a density-based clustering algorithm based on rectangular distance is used to cluster the second set of filtering prediction boxes into multiple clusters. Specifically, this includes: setting an unvisited marker, a distance range parameter, and a threshold for the number of neighboring points for each second filtering prediction box in the set; sequentially traversing each second filtering prediction box in the set, modifying the unvisited marker of the current second filtering prediction box to a visited marker; if the number of first neighboring prediction boxes within the distance range parameter corresponding to the current second filtering prediction box is not less than the threshold for the number of neighboring points, then the current second filtering prediction box is used as a core point, and a first cluster is created based on the core point; all first neighboring prediction boxes corresponding to the current second filtering prediction box are added to a waiting queue; if the waiting queue is not empty, any first neighboring prediction box is selected from the waiting queue. As the target first neighborhood prediction box, if the marker corresponding to the target first neighborhood prediction box is an unvisited marker, then the unvisited marker of the target first neighborhood prediction box is modified to the visited marker; obtain all second neighborhood prediction boxes corresponding to the target first neighborhood prediction box; if the number of second neighborhood prediction boxes corresponding to all second neighborhood prediction boxes is not less than the neighborhood point number threshold, add all second neighborhood prediction boxes to the processing queue; check whether the target first neighborhood prediction box has been assigned to a cluster; if not, assign the target first neighborhood prediction box from the processing queue to the first cluster, and re-determine whether the processing queue is empty; if the processing queue is empty, complete the creation of the first cluster; check whether each second filtering prediction box in the second filtering prediction box set has been traversed; if not, re-traverse each second filtering prediction box in the second filtering prediction box set; otherwise, integrate all first clusters to obtain multiple clusters.

[0011] In one possible implementation, the process of detecting the carriage in an aluminum image to determine the carriage region image specifically includes: inputting the aluminum image to be detected into a pre-trained carriage detection model, so that the carriage detection model performs carriage detection on the aluminum image and outputs a carriage prediction box and the carriage confidence score corresponding to the carriage prediction box; comparing the carriage confidence score with a preset carriage confidence threshold, and if the carriage confidence score is greater than the preset carriage confidence threshold, then retaining the carriage prediction box and using the image region corresponding to the carriage prediction box as the carriage region image; wherein, the pre-training process of the carriage detection model includes: based on an edge detection algorithm, detecting each loading material sample image in the collected loading material sample image set, extracting the carriage candidate connected regions in the loading material sample images, and calculating the carriage candidate connected regions. The minimum bounding rectangle is used to determine the first truck compartment prediction box, and the first truck compartment prediction box is labeled to obtain a truck compartment labeled image dataset. The truck compartment labeled image dataset is input into the YOLOv8 model, so that the YOLOv8 model extracts multi-scale features of the image through the backbone network for each truck compartment labeled image in the truck compartment labeled image dataset, performs feature fusion of the multi-scale features of the image through the neck network, and outputs the model prediction result by the head network. The model prediction result includes the coordinates of the truck compartment prediction box and the confidence of the truck compartment prediction box corresponding to the coordinates. The loss value between the model prediction result and the real result is calculated. Based on the loss value, the backpropagation algorithm is used to backpropagate the model parameters of the YOLOv8 model to obtain the optimized truck compartment detection model.

[0012] In one possible implementation, the model training process of the profile detection model specifically includes: acquiring a set of sample images of a regular carriage area and a set of sample images of an abnormal carriage area; training an initial profile detection model with a first learning rate based on the set of sample images of the regular carriage area to obtain a first profile detection model; training the first profile detection model with a second learning rate based on the set of sample images of the abnormal carriage area to obtain a second profile detection model; and training the second profile detection model with a third learning rate based on the set of sample images of the regular carriage area and the set of sample images of the abnormal carriage area to obtain a profile detection model.

[0013] Secondly, this application provides an automatic profile detection device, comprising: a carriage detection module, a prediction box filtering module, and a counting module; wherein, the carriage detection module is used to perform carriage detection on the image of the loaded profile, determine the carriage area image, and input the carriage area image into a pre-trained profile detection model, so that the profile detection model performs profile detection on the carriage area image and outputs multiple profile prediction boxes; the prediction box filtering module is used to perform a first filtering process on the multiple profile prediction boxes based on the prediction box morphology features to obtain a first set of filtered prediction boxes, a second filtering process on the first set of filtered prediction boxes based on the prediction box position features to obtain a second set of filtered prediction boxes, and a third filtering process on the second set of filtered prediction boxes based on the prediction box distance features to obtain a third set of filtered prediction boxes; the counting module is used to count the third set of filtered prediction boxes to obtain the number of profiles in the aluminum material image.

[0014] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0016] This application provides an automatic profile inspection method, apparatus, device, and storage medium, which has the following advantages compared with the prior art:

[0017] By performing vehicle compartment detection on the image of the loading materials, the vehicle compartment area image is determined. This image is then input into a pre-trained profile detection model, which detects profiles within the vehicle compartment area and outputs multiple profile prediction boxes. Based on the shape features of the prediction boxes, a first set of filtered prediction boxes is obtained. Based on the position features of the prediction boxes, this first set is further filtered to obtain a second set. Based on the distance features of the prediction boxes, this second set is further filtered to obtain a third set. Finally, the third set of prediction boxes is counted to determine the number of profiles in the loading material image. Compared with existing technologies, this application's technology... The proposed solution performs vehicle compartment detection on images of materials being loaded onto trucks, accurately delineating the compartment area and effectively eliminating external interference, significantly improving the detection's anti-interference capability. It then utilizes a pre-trained material detection model to detect the compartment area image, outputting predicted bounding boxes. Based on the shape, position, and distance features of these predicted boxes, a three-stage filtering process is performed to optimize the detection results, effectively avoiding false positives and false negatives caused by scene changes in traditional methods. This greatly enhances the method's robustness and enables accurate detection of small-target materials even in dense scenes. Finally, the filtered predicted boxes are counted to obtain the accurate number of materials. The entire process eliminates the need for complex feature extraction, reducing reliance on computational resources and achieving efficient and accurate automatic detection and counting of materials in industrial loading scenarios. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0021] Figure 1 This is a flowchart illustrating one embodiment of the automatic profile inspection method provided in this application;

[0022] Figure 2This is a schematic diagram of one embodiment of the automatic profile inspection device provided in this application;

[0023] Figure 3 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0026] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0027] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0030] Example 1, see Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the automatic profile inspection method provided in this application, as shown below. Figure 1 As shown, the method includes steps 101-103, as detailed below:

[0031] Step 101: Perform car body detection on the loading material image to determine the car body area image. Input the car body area image into the pre-trained material detection model so that the material detection model can perform material detection on the car body area image and output multiple material prediction boxes.

[0032] In one embodiment, in an industrial loading scenario, the profiles are only loaded inside the truck bed, and the truck bed area is the specific space where the profiles exist. The loading profile image contains a large amount of information, including not only the profiles but also interfering elements such as the ground outside the truck bed, other vehicles, and debris. Therefore, in this embodiment, the truck bed detection is first performed on the image based on a pre-trained profile detection model, which can accurately delineate the truck bed area and focus the subsequent profile detection range on the inside of the truck bed, avoiding interference from the external environment of the truck bed, improving the accuracy and robustness of the detection. Compared with directly detecting the entire loading profile image, this can significantly reduce the amount of data that needs to be processed, reduce computational complexity, and thus significantly improve detection efficiency.

[0033] Preferably, the profile includes, but is not limited to, aluminum profiles.

[0034] In one embodiment, the pre-training process of the truck compartment detection model includes: based on an edge detection algorithm, detecting each truck loading material sample image in the collected truck loading material sample image set, extracting candidate connected regions of the truck compartment in the truck loading material sample image, calculating the minimum bounding rectangle of the candidate connected regions of the truck compartment, determining the first truck compartment prediction box, and annotating the first truck compartment prediction box to obtain a truck compartment annotated image dataset; inputting the truck compartment annotated image dataset into a YOLOv8 model, so that the YOLOv8 model extracts multi-scale features of the image through the backbone network for each truck compartment annotated image in the truck compartment annotated image dataset, performs feature fusion of the multi-scale features of the image through the neck network, and outputs the model prediction result by the head network, wherein the model prediction result includes the coordinates of the truck compartment prediction box and the confidence score of the truck compartment prediction box corresponding to the coordinates of the truck compartment prediction box; calculating the loss value between the model prediction result and the real result, and based on the loss value, using the backpropagation algorithm to perform backpropagation optimization processing on the model parameters of the YOLOv8 model to obtain an optimized truck compartment detection model.

[0035] Specifically, the YOLOv8 model is selected for model training to construct a carriage detection model; wherein, the YOLOv8 model includes a backbone network, a neck network, and a head network.

[0036] Specifically, the backbone network consists of multiple convolutional layers and is mainly used to extract image features. It learns features under different receptive fields based on different model layers, that is, it extracts feature maps of different scales from the input image. The neck network mainly fuses feature maps of different scales from the backbone network to enhance the model's feature representation ability, especially when dealing with objects of different sizes. The head network is the final prediction part of the model and is responsible for generating the final object detection result based on the feature map output by the neck network, including bounding box coordinates, object confidence, and class probability.

[0037] Specifically, before model training, a high-quality dataset of labeled truck cargo images needs to be constructed. First, a large number of loading material sample images containing trucks are collected to form a loading material sample image set. Edge detection algorithms such as Canny are used to process the loading material sample images in the loading material sample image set. Based on prior knowledge of the cargo size (usually occupying the main area of ​​the image), cargo shape (rectangular or quadrilateral), and cargo position (located in the center of the image), candidate connected regions of the cargo are extracted. The first cargo prediction box is determined by calculating the minimum bounding rectangle of the candidate connected regions of the cargo. After manual review and correction, an accurate truck cargo labeled image dataset is formed. This semi-automated labeling method greatly improves the efficiency of data preparation and is particularly suitable for processing large batches of images in industrial scenarios.

[0038] Specifically, the constructed dataset of labeled truck cargo images is input into the YOLOv8 model. The backbone network of the YOLOv8 model extracts multi-scale features from each labeled truck cargo image in the dataset. These multi-scale features cover various information about the cargo in the image, from local details to the overall outline. The extracted multi-scale features are then input into the neck network for fusion processing, enabling the model to integrate information at different scales and more comprehensively understand the features of the cargo in the image. Finally, the head network outputs the model prediction results based on the fused features. The model prediction results include the coordinates of the predicted cargo box and the confidence score of the predicted cargo box corresponding to the coordinates. The confidence score reflects the degree of certainty of the model regarding the detection results.

[0039] Specifically, to make the model's predictions more closely resemble reality, a loss value is calculated between the model's predictions and the actual labeled results. This loss value measures the degree of prediction bias. Based on the loss value, the YOLOv8 model parameters are optimized using the backpropagation algorithm. The backpropagation algorithm propagates the loss value backward along the neural network layers, adjusting the parameters of each layer according to the loss. Through iterative training, the loss value is gradually reduced, continuously improving the model's prediction accuracy. Ultimately, an optimized carriage detection model is obtained, used for the accurate detection of carriage areas in aluminum material images.

[0040] In one embodiment, the image of the aluminum material to be detected is input into a pre-trained carriage detection model, so that the carriage detection model performs carriage detection on the aluminum material image and outputs a carriage prediction box and the carriage confidence score corresponding to the carriage prediction box; the carriage confidence score is compared with a preset carriage confidence threshold. If the carriage confidence score is greater than the preset carriage confidence threshold, the carriage prediction box is retained, and the image region corresponding to the carriage prediction box is used as the carriage region image.

[0041] Specifically, when detecting the carriage in an aluminum profile image, the image is input into a pre-trained carriage detection model. The model analyzes the image, using learned features and algorithms to identify the carriage's location within the image and output a predicted carriage bounding box. Simultaneously, the model provides a carriage confidence score for each predicted box, measuring the model's reliability. A higher score indicates greater confidence in the detected carriage area. This confidence score is compared to a pre-set threshold. If the confidence score exceeds this threshold, the detection result is considered highly reliable, and the predicted carriage bounding box is retained. The image area within the predicted box is then designated as the carriage region image, serving as the target area for subsequent profile detection and counting.

[0042] Preferably, if the model outputs multiple carriage prediction boxes, first obtain the confidence scores corresponding to all carriage prediction boxes, and retain the carriage prediction box with the highest confidence score as the baseline carriage prediction box; remove other carriage prediction boxes whose overlap with the baseline carriage prediction box exceeds a preset threshold, and finally retain only one carriage prediction box with the highest confidence score and the best coverage. Then compare the confidence score corresponding to the retained carriage prediction box with the preset carriage confidence score threshold. If the confidence score corresponding to the carriage prediction box is not less than the preset carriage confidence score threshold, it means that the model's judgment that the carriage prediction box is a carriage is reliable enough, and the carriage prediction box is retained, and the image area within the carriage prediction box is determined as the carriage area image; if the confidence score corresponding to the carriage prediction box is less than the preset carriage confidence score threshold, it means that the detection result may be a false detection, and the carriage prediction box is discarded, and it is regarded as having no valid carriage area in the image.

[0043] In one embodiment, after determining the carriage area image in the model material image, the carriage area image is input into a pre-trained model material detection model for further aluminum material detection processing.

[0044] In one embodiment, the model training process of the profile detection model includes: acquiring a set of sample images of a regular carriage area and a set of sample images of an abnormal carriage area; training an initial profile detection model with a first learning rate based on the set of sample images of the regular carriage area to obtain a first profile detection model; training the first profile detection model with a second learning rate based on the set of sample images of the abnormal carriage area to obtain a second profile detection model; and training the second profile detection model with a third learning rate based on the set of sample images of the regular carriage area and the set of sample images of the abnormal carriage area to obtain a profile detection model.

[0045] Specifically, the set of sample images of the regular carriage area consists of multiple sample images of carriage areas with undamaged or unobstructed profiles; the set of sample images of the abnormal carriage area consists of sample images of carriage areas with damaged or obstructed profiles or loose packaging.

[0046] Specifically, since the number of profiles in the loading scenario often exceeds 300, manual annotation is extremely costly. Therefore, in this embodiment, based on the SAM model, pre-annotation processing is performed on each car area sample image in the normal car area sample image set and the abnormal car area sample image set, and the annotation of the profile prediction box corresponding to each car area sample image is automatically generated. The SAM model can improve annotation efficiency and significantly reduce manual workload through zero-sample segmentation capability. At the same time, based on manual correction of obvious errors in SAM annotation, such as missegmentation of damaged areas and missed detection of overlapping profiles, the SAM pre-annotation combined with manual correction is used to further correct and improve the normal car area sample image set and the abnormal car area sample image set.

[0047] Specifically, in the first stage of model training, the initial profile detection model composed of YOLOv8 model is trained based on the sample image set of the regular carriage area with a large first learning rate to obtain the first profile detection model. The goal of this stage is to enable the model to quickly learn the basic visual features of the profile, such as shape, color, and texture, and establish an efficient detection capability for normal profiles. Due to the large learning rate, the model can converge quickly.

[0048] Specifically, in the second stage of model training, the first profile detection model trained in the first stage is fine-tuned using a smaller second learning rate based on the sample image set of abnormal carriage areas, resulting in the second profile detection model. The purpose of this stage is to enable the backbone network of the model to gradually adapt to the complex situation of abnormal samples and learn the features of occluded and damaged profiles while minimizing the disruption of the original normal sample features. A small learning rate can prevent the model from forgetting the normal sample features it has learned before, while also enhancing its robustness to abnormal samples.

[0049] Specifically, in the third stage of model training, the second profile detection model trained in the second stage is globally fine-tuned based on the sample image set of the normal carriage area and the sample image set of the abnormal carriage area using a very small third learning rate, to obtain the profile detection model. The role of this stage is to balance the model's ability to identify normal and abnormal samples, ensuring that the model can accurately detect normal profiles and handle complex situations such as damage and occlusion. The very small learning rate can prevent the model from overfitting and optimize the overall detection performance.

[0050] Specifically, the first learning rate is greater than the second learning rate, and the second learning rate is greater than the third learning rate.

[0051] Specifically, since the initial set of sample images of the regular carriage area and the sample image set of the abnormal carriage area may have annotation errors, the profile detection model obtained after the above training automatically corrects the sample image sets of the regular carriage area and the sample image set of the abnormal carriage area, such as adjusting the position of the prediction box in each image or supplementing the missing aluminum material; through multiple training-correction annotation-retraining iterations, the data quality and model accuracy are continuously improved.

[0052] Step 102: Based on the shape features of the prediction boxes, perform a first filtering process on the multiple profile prediction boxes to obtain a first set of filtered prediction boxes. Based on the position features of the prediction boxes, perform a second filtering process on the first set of filtered prediction boxes to obtain a second set of filtered prediction boxes. Based on the distance features of the prediction boxes, perform a third filtering process on the second set of filtered prediction boxes to obtain a third set of filtered prediction boxes.

[0053] In one embodiment, since the features of some areas and gaps in the truck compartment are very similar to those of aluminum materials with damaged packaging in the loading scenario, false detection is likely to occur. Therefore, in this embodiment, for the special case of loading scenario, the multiple profile prediction frames output by the profile detection model are screened in three rounds based on the prediction frame shape features, prediction frame position features and prediction frame distance features to handle false detections with other interference items, so as to correct the detection results.

[0054] In one embodiment, based on the shape characteristics of the prediction frames, a first filtering process is performed on the plurality of profile prediction frames to obtain a first set of filtered prediction frames. This is achieved by acquiring the shape characteristics of each of the plurality of profile prediction frames, wherein the shape characteristics include prediction frame length, prediction frame width, prediction frame area, and prediction frame aspect ratio. The process involves determining whether the prediction frame length, width, and area are within a preset range, and whether the prediction frame aspect ratio is within a preset range. If any prediction frame length is outside the preset range, or any prediction frame width is outside the preset range, the process is repeated. If the predicted frame area is not within the preset width range, or the predicted frame aspect ratio is not within the preset aspect ratio range, then the profile predicted frame corresponding to the current predicted frame shape feature is removed. Based on the predicted frame shape feature, the mean and standard deviation of the predicted frame shape feature are calculated, and a normal distribution confidence interval is constructed based on the mean and standard deviation of the predicted frame shape feature. It is determined whether the current predicted frame shape feature exceeds the normal distribution confidence interval. If so, the profile predicted frame corresponding to the current predicted frame shape feature is removed. All the remaining profile predicted frames are integrated to obtain the first set of filtered predicted frames.

[0055] Specifically, based on historical experience, the upper and lower limits corresponding to the shape features of the prediction box are determined, and based on the upper and lower limits, the preset length range, preset width range, preset area range and preset aspect ratio range are determined respectively.

[0056] Specifically, by comparing the shape features of multiple profile prediction frames with their corresponding preset ranges, if any feature exceeds the range, it indicates that the target shape corresponding to the prediction frame differs too much from the standard profile, and it is directly rejected.

[0057] Specifically, according to the central limit theorem, when any distribution is sampled a sufficient number of times, its sample mean will approximately follow a normal distribution. Therefore, in this embodiment, by assuming that the shape features of the predicted box, such as its length, width, area, and aspect ratio, follow a normal distribution, and then using the property that the sample of the normal distribution has a 99.73% probability of falling on it, the various attributes of the box are screened again.

[0058] Specifically, by calculating the mean value μ of the morphological features of the prediction box and the standard deviation σ of the mean value of the morphological features of the prediction box, a normal distribution confidence interval [μ - 3σ, μ + 3σ] is constructed based on the mean value of the morphological features of the prediction box and the standard deviation of the mean value of the morphological features of the prediction box; when the morphological features of the prediction box are not within the normal distribution confidence interval, it indicates that the length of this prediction box is too long or too short, not meeting the size of a normal prediction box, and then this prediction box will be discarded; after the above screening, those prediction boxes with too large or too small shapes or proportions will be processed.

[0059] In one embodiment, when performing a secondary screening process on the first screened prediction box set based on the position features of the prediction box to obtain a second screened prediction box set, by obtaining the coordinates of the first screened prediction box corresponding to each first screened prediction box in the first screened prediction box set, comparing the coordinates of the first screened prediction box with the boundaries of the rectangular box corresponding to the carriage area image, if the first screened prediction box is located outside the rectangular box boundary, the current first screened prediction box is removed, and if the first screened prediction box is located within the rectangular box boundary, the current first screened prediction box is retained; and obtaining the first vertical coordinate corresponding to each first screened prediction box in the first screened prediction box set, and based on the first vertical coordinate, performing a descending order sorting and hierarchical processing on each first screened prediction box to assign each first screened prediction box to a matching first screened prediction box layer; traversing each first screened prediction box in the first screened prediction box set in layer order from bottom to top to obtain the number of prediction boxes in multiple layers, and respectively comparing each number of prediction boxes in a layer with a preset threshold value of the number of prediction boxes in a layer, if the number of prediction boxes in a layer is less than the preset threshold value of the number of prediction boxes in a layer, all the first screened prediction boxes in the first screened prediction box layer corresponding to the number of prediction boxes in a layer are removed; integrating all the remaining first screened prediction boxes to obtain a second screened prediction box set.

[0060] Specifically, obtain the coordinates of the i-th first screened prediction box and the boundary coordinates (X1, Y1, X2, Y2) of the rectangular box corresponding to the carriage prediction box, and determine whether any of the following conditions is met: x1 < X1 or x2 > X2 (exceeding the left and right boundaries of the carriage); y1 < Y1 or y2 > Y2 (exceeding the upper and lower boundaries of the carriage). If so, it is determined that the carriage prediction box is located outside the carriage and is directly removed.

[0061] Preferably, the coordinate system corresponding to the coordinates mentioned in the embodiments of the present application is an image coordinate system, and the origin of the image coordinate system is at the upper left corner, the positive x-axis direction is to the right, and the positive y-axis direction is downward.

[0062] Specifically, the bottom of the carriage often contains interference such as floor patterns and fixing devices, which can easily be misidentified as profiles. For example, the weld seam of the floor may form a slender prediction frame, which may still remain in the bottom area of ​​the carriage after boundary filtering. Therefore, some prediction frames located at the bottom of the carriage need to be further screened.

[0063] Specifically, since the origin of the image coordinate system is the top left corner, the images are sorted in descending order according to the first y-coordinate of the first filtering prediction box, which corresponds to the order in the real world from the bottom of the carriage (large y-coordinate) to the top of the carriage (small y-coordinate).

[0064] Specifically, since the profiles are horizontally stacked in industrial loading, each layer should contain an approximate number of profiles. Based on this, in the embodiments of this application, the sorted first screening prediction box is further divided into multiple layers to obtain multiple first screening prediction box layers.

[0065] Specifically, multiple first-filter prediction box layers are checked layer by layer in order from bottom to top. The bottom layer is processed first, and then the upper layer is processed. If the number of prediction boxes in a certain first-filter prediction box layer is less than the preset threshold for the number of prediction boxes in a layer, it indicates that the layer may be noise, and all first-filter prediction boxes in the layer are removed. If the number meets the threshold, all first-filter prediction boxes in the layer are retained.

[0066] Specifically, when profiles are densely stacked, the first ordinate of adjacent prediction boxes may be close. Traditional layering, such as fixed pixel intervals, can easily lead to the same pile of profiles being split into different layers. Therefore, in this embodiment, the mean of the first ordinate of all first screening prediction boxes and the standard deviation of the first ordinate are calculated. When the difference of the first ordinate of adjacent first screening prediction boxes is less than half of the standard deviation of the first ordinate, the current first screening prediction box and the adjacent first screening prediction box belong to the same layer. The layer is dynamically generated by using a disjoint-set data structure and first screening prediction boxes that meet the conditions, thereby improving the layering accuracy.

[0067] In one embodiment, based on the distance feature of the prediction boxes, the second set of filtered prediction boxes is subjected to three filtering processes to obtain the third set of filtered prediction boxes. Then, a density clustering algorithm based on rectangular distance is used to cluster the second set of filtered prediction boxes to divide it into multiple clusters. The largest cluster among the multiple clusters is determined, and all second-filtered prediction boxes corresponding to non-largest clusters are removed, while all second-filtered prediction boxes corresponding to the largest cluster are retained. All the retained second-filtered prediction boxes are integrated to obtain the third set of filtered prediction boxes.

[0068] In one embodiment, a density clustering algorithm based on rectangular distance is used to cluster the second set of filtering prediction boxes to divide the second set of filtering prediction boxes into multiple clusters. This is achieved by setting an unvisited flag, a distance range parameter, and a threshold for the number of neighboring points for each second filtering prediction box in the second set of filtering prediction boxes. Each second filtering prediction box in the second set of filtering prediction boxes is traversed sequentially, and the unvisited flag of the current second filtering prediction box is modified to a visited flag. If the number of first neighboring prediction boxes within the distance range parameter corresponding to the current second filtering prediction box is not less than the threshold for the number of neighboring points, then the current second filtering prediction box is taken as a core point, and a first cluster is created based on the core point. All first neighboring prediction boxes corresponding to the current second filtering prediction box are added to a waiting queue. If the waiting queue is not empty, any first neighboring prediction box is selected from the waiting queue as the target cluster. First, a first neighbor prediction box is marked. If the marker corresponding to the first neighbor prediction box is an unvisited marker, the unvisited marker of the first neighbor prediction box is changed to the visited marker. All second neighbor prediction boxes corresponding to the first neighbor prediction box are obtained. If the number of second neighbor prediction boxes corresponding to all second neighbor prediction boxes is not less than the threshold number of neighbor points, all second neighbor prediction boxes are added to the processing queue. It is checked whether the first neighbor prediction box has been assigned to a cluster. If not, the first neighbor prediction box is assigned from the processing queue to the first cluster, and the processing queue is re-determined to be empty. If the processing queue is empty, the creation of the first cluster is completed. It is checked whether each second filter prediction box in the second filter prediction box set has been traversed. If not, each second filter prediction box in the second filter prediction box set is traversed again; otherwise, all first clusters are integrated to obtain multiple clusters.

[0069] Specifically, three key parameters are set for each second filtering prediction box in the second filtering prediction box set, including an access flag, a distance range parameter, and a neighboring point number threshold. The access flag includes an unvisited flag and a visited flag, used to record whether the second filtering prediction box has been processed. This flag is dynamically updated during traversal to avoid duplicate processing of prediction boxes. The distance range parameter is a preset neighborhood radius, used to measure the threshold distance between two prediction boxes. If the rectangular distance between two prediction boxes is less than this distance range parameter, they are considered spatially close. The neighboring point number threshold is a preset integer value, used to determine that when the number of first neighboring prediction boxes within the distance range parameter of a prediction box reaches or exceeds the neighboring point number threshold, the prediction box is considered to be in a densely populated profile area.

[0070] Specifically, each second filter prediction box in the second filter prediction box set is accessed sequentially, and the access mark of the currently accessed second filter prediction box is changed from the unaccessed mark to the accessed mark to prevent repeated processing of the second filter prediction box in the future.

[0071] Specifically, for the current second filtering prediction box, check the number of first neighbor prediction boxes within its distance range parameter; if the number of first neighbor prediction boxes is not less than the threshold of the number of neighbor points, then the current second filtering prediction box is identified as the core point, and a new cluster is created based on it, starting a new clustering branch.

[0072] Specifically, traditional density clustering algorithms use the distance between points, such as Euclidean distance and Manhattan distance, to determine the neighborhood relationship of sample points; however, in the profile inspection scenario, since the inspection results exist in the form of rectangular prediction boxes, it is necessary to define a distance measurement method between the rectangles to more accurately reflect the spatial positional relationship of the profiles.

[0073] Specifically, when checking the number of first neighboring predicted boxes within the distance range parameter of the current second filtering predicted box, the coordinates of the second filtering predicted box corresponding to the current second filtering predicted box and the coordinates of the first predicted box of the selected adjacent predicted box to be determined are obtained. The coordinates of the second filtering predicted box and the coordinates of the first predicted box are substituted into the rectangular box distance measurement formula to obtain the rectangular distance between the current second filtering predicted box and the adjacent predicted box to be determined. It is then determined whether the rectangular distance is within the distance range parameter. If so, the adjacent predicted box to be determined is the first neighboring predicted box within the distance range parameter of the current second filtering predicted box. The rectangular box distance measurement formula is as follows:

[0074]

[0075] In the formula, D is the rectangular distance, x 1,min Let x be the x-coordinate of the top-left corner of the first rectangle. 1,max Let x be the x-coordinate of the bottom right corner of the first rectangle. 2,min Let x be the x-coordinate of the top-left corner of the second rectangle. 2,max The x-coordinate and y-coordinate of the bottom right corner of the second rectangle are... 1,min The y-coordinate of the top-left corner of the first rectangle is y. 1,max The y-coordinate of the bottom right corner of the first rectangle is y. 2,min The y-coordinate of the top-left corner of the second rectangle is y. 2,max Let d be the y-coordinate of the bottom right corner of the second rectangle. x Let d be the distance between the two rectangles along the x-axis. y This represents the distance between the two rectangles along the y-axis. Let be the Euclidean distance between the two rectangles.

[0076] Specifically, all first-neighbor prediction boxes of the core points are added to the processing queue, which serves as a temporary storage area for the neighbor prediction boxes to be analyzed. When the processing queue is not empty, a prediction box is taken from the processing queue as the target first-neighbor prediction box. If the access flag of the target first-neighbor prediction box is unvisited, its flag status is changed to visited, and the number of its own second-neighbor prediction boxes is checked. If the number of second-neighbor prediction boxes reaches the threshold of the number of neighbor points, all second-neighbor prediction boxes are added to the processing queue to continue expanding the clustering range. At the same time, if the target first-neighbor prediction box has not yet been assigned to any cluster, it is assigned to the first cluster currently being processed. The above steps are repeated until the processing queue is empty, at which point the construction of a cluster is completed.

[0077] Specifically, after traversing all the second filter prediction boxes in the second filter prediction box set, all the generated clusters are integrated to obtain the clustering results, and the largest cluster is selected from the clustering results to obtain the third filter prediction box set.

[0078] Step 103: Perform counting processing on the third set of filtering prediction boxes to obtain the number of profiles in the aluminum material image.

[0079] In one embodiment, the third set of prediction frames is directly counted, and the calculated number of prediction frames is used as the number of profiles in the aluminum image.

[0080] In one embodiment, based on the automatic profile detection method given in the embodiments of this application, multiple images of vehicle-mounted profile samples are subjected to profile detection. After multiple profile prediction boxes output by the profile detection model are filtered through the three-round filtering method provided in the above embodiments, the counting error rate between the predicted number of profiles and the actual number of profiles is calculated. The calculation error rate is within 1%, thereby achieving efficient and accurate automatic profile detection and counting.

[0081] Specifically, the formula for calculating the counting error rate is as follows:

[0082]

[0083] Where, n p To predict the number of profiles, n t represents the actual number of profiles, and abs(·) is the absolute value function.

[0084] Example 2, see Figure 2 , Figure 2This is a schematic diagram of an embodiment of an automatic profile inspection device provided in this application. Corresponding to the above-described automatic profile inspection method, this application also provides an automatic profile inspection device. This automatic profile inspection device includes modules for executing the above-described automatic profile inspection method, and can be configured in a desktop computer, tablet computer, laptop computer, or other terminal; specifically, the automatic profile inspection device includes a carriage inspection module 201, a prediction frame filtering module 202, and a counting module 203.

[0085] The carriage detection module 201 is used to perform carriage detection on the loading material image, determine the carriage area image, and input the carriage area image into the pre-trained profile detection model so that the profile detection model can perform profile detection on the carriage area image and output multiple profile prediction boxes.

[0086] The prediction frame filtering module 202 is used to perform a first filtering process on the multiple profile prediction frames based on the prediction frame shape characteristics to obtain a first set of filtered prediction frames, perform a second filtering process on the first set of filtered prediction frames based on the prediction frame position characteristics to obtain a second set of filtered prediction frames, and perform a third filtering process on the second set of filtered prediction frames based on the prediction frame distance characteristics to obtain a third set of filtered prediction frames.

[0087] The counting module 203 is used to perform counting processing on the third set of filtering prediction boxes to obtain the number of profiles in the aluminum material image.

[0088] In one embodiment, the prediction frame filtering module 202 is used to perform a filtering process on the plurality of profile prediction frames based on the prediction frame shape features to obtain a first set of filtered prediction frames. Specifically, this includes: obtaining the prediction frame shape features corresponding to each of the plurality of profile prediction frames, wherein the prediction frame shape features include prediction frame length, prediction frame width, prediction frame area, and prediction frame aspect ratio; determining whether the prediction frame length is within a preset length range, determining whether the prediction frame width is within a preset width range, determining whether the prediction frame area is within a preset area range, and determining whether the prediction frame aspect ratio is within a preset aspect ratio range; if any prediction frame length is not within the preset length range, If the width of the prediction frame is not within the preset width range, the area of ​​the prediction frame is not within the preset area range, or the aspect ratio of the prediction frame is not within the preset aspect ratio range, then the profile prediction frame corresponding to the current prediction frame shape feature is removed. Based on the prediction frame shape feature, the mean and standard deviation of the prediction frame shape feature are calculated, and a normal distribution confidence interval is constructed based on the mean and standard deviation of the prediction frame shape feature. It is then determined whether the current prediction frame shape feature exceeds the normal distribution confidence interval. If so, the profile prediction frame corresponding to the current prediction frame shape feature is removed. All the remaining profile prediction frames are integrated to obtain the first set of filtered prediction frames.

[0089] In one embodiment, the prediction box filtering module 202 is used to perform a secondary filtering process on the first set of filtered prediction boxes based on the prediction box position features to obtain a second set of filtered prediction boxes. Specifically, this includes: obtaining the coordinates of the first filtered prediction box corresponding to each first filtered prediction box in the first set of filtered prediction boxes; comparing the coordinates of the first filtered prediction box with the boundary of the rectangle corresponding to the carriage area image; if the first filtered prediction box is located outside the boundary of the rectangle, then the current first filtered prediction box is removed; if the first filtered prediction box is located inside the boundary of the rectangle, then the current first filtered prediction box is retained; and obtaining the coordinates of each first filtered prediction box in the first set of filtered prediction boxes. Based on the first ordinate corresponding to the frame, each first filtering prediction frame is sorted in descending order and layered to assign each first filtering prediction frame to a matching first filtering prediction frame layer. The first filtering prediction frame in the set is traversed layer by layer from bottom to top to obtain multiple layers of prediction frames. The number of prediction frames in each layer is compared with a preset threshold. If the number of prediction frames in a layer is less than the preset threshold, all first filtering prediction frames in the first filtering prediction frame layer corresponding to that number are removed. All the remaining first filtering prediction frames are then integrated to obtain a second filtering prediction frame set.

[0090] In one embodiment, the prediction box filtering module 202 is used to perform three filtering processes on the second set of filtered prediction boxes based on the distance features of the prediction boxes to obtain a third set of filtered prediction boxes. Specifically, it includes: using a density clustering algorithm based on rectangular distance to cluster the second set of filtered prediction boxes to divide the second set of filtered prediction boxes into multiple clusters; determining the largest cluster among the multiple clusters, removing all second-filtered prediction boxes corresponding to the non-largest cluster among the multiple clusters, and retaining all second-filtered prediction boxes corresponding to the largest cluster; and integrating all the retained second-filtered prediction boxes to obtain the third set of filtered prediction boxes.

[0091] In one embodiment, the prediction box filtering module 202 is used to cluster the second set of filtered prediction boxes using a density clustering algorithm based on rectangular distance to divide the second set of filtered prediction boxes into multiple clusters. Specifically, this includes: setting a corresponding unvisited label, distance range parameter, and neighborhood point number threshold for each second filtered prediction box in the second set of filtered prediction boxes; sequentially traversing each second filtered prediction box in the second set of filtered prediction boxes, modifying the unvisited label of the current second filtered prediction box to a visited label; if the number of first neighborhood prediction boxes within the distance range parameter corresponding to the current second filtered prediction box is not less than the neighborhood point number threshold, then the current second filtered prediction box is taken as a core point, and a first cluster is created based on the core point; all first neighborhood prediction boxes corresponding to the current second filtered prediction box are added to a waiting queue; if the waiting queue is not empty, any first neighborhood prediction box is selected from the waiting queue. The neighborhood prediction box is used as the target first neighborhood prediction box. If the marker corresponding to the target first neighborhood prediction box is an unvisited marker, then the unvisited marker of the target first neighborhood prediction box is modified to the visited marker. All second neighborhood prediction boxes corresponding to the target first neighborhood prediction box are obtained. If the number of second neighborhood prediction boxes corresponding to all second neighborhood prediction boxes is not less than the neighborhood point number threshold, all second neighborhood prediction boxes are added to the waiting queue. It is checked whether the target first neighborhood prediction box has been assigned to a cluster. If not, the target first neighborhood prediction box is assigned from the waiting queue to the first cluster, and the waiting queue is re-determined to be empty. If the processing queue is empty, the creation of the first cluster is completed. It is checked whether each second filtering prediction box in the second filtering prediction box set has been traversed. If not, each second filtering prediction box in the second filtering prediction box set is traversed again. Otherwise, all first clusters are integrated to obtain multiple clusters.

[0092] In one embodiment, the carriage detection module 201 is used to perform carriage detection on the aluminum material image to determine the carriage region image. Specifically, it includes: inputting the aluminum material image to be detected into a pre-trained carriage detection model, so that the carriage detection model performs carriage detection on the aluminum material image and outputs a carriage prediction box and the carriage confidence score corresponding to the carriage prediction box; comparing the carriage confidence score with a preset carriage confidence score threshold; if the carriage confidence score is greater than the preset carriage confidence score threshold, then the carriage prediction box is retained, and the image region corresponding to the carriage prediction box is taken as the carriage region image.

[0093] In one embodiment, the pre-training process of the truck compartment detection model includes: based on an edge detection algorithm, detecting each truck loading material sample image in the collected truck loading material sample image set, extracting candidate connected regions of the truck compartment in the truck loading material sample image, calculating the minimum bounding rectangle of the candidate connected regions of the truck compartment, determining the first truck compartment prediction box, and annotating the first truck compartment prediction box to obtain a truck compartment annotated image dataset; inputting the truck compartment annotated image dataset into a YOLOv8 model, so that the YOLOv8 model extracts multi-scale features of the image through the backbone network for each truck compartment annotated image in the truck compartment annotated image dataset, performs feature fusion of the multi-scale features of the image through the neck network, and outputs the model prediction result by the head network, wherein the model prediction result includes the coordinates of the truck compartment prediction box and the confidence score of the truck compartment prediction box corresponding to the coordinates of the truck compartment prediction box; calculating the loss value between the model prediction result and the real result, and based on the loss value, using the backpropagation algorithm to perform backpropagation optimization processing on the model parameters of the YOLOv8 model to obtain an optimized truck compartment detection model.

[0094] In one embodiment, the model training process of the profile detection model specifically includes: acquiring a set of sample images of a regular carriage area and a set of sample images of an abnormal carriage area; training an initial profile detection model with a first learning rate based on the set of sample images of the regular carriage area to obtain a first profile detection model; training the first profile detection model with a second learning rate based on the set of sample images of the abnormal carriage area to obtain a second profile detection model; and training the second profile detection model with a third learning rate based on the set of sample images of the regular carriage area and the set of sample images of the abnormal carriage area to obtain a profile detection model.

[0095] The above-described automatic profile inspection device can implement the automatic profile inspection method of the above method embodiments. The options in the above method embodiments are also applicable to this embodiment, and will not be described in detail here.

[0096] like Figure 3 As shown, Figure 3This is a schematic diagram of the structure of a computer device provided in this application; it includes a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 communicate with each other through the communication bus 114, and the memory 113 is used to store computer programs.

[0097] In one embodiment of this application, the processor 111, when executing the program stored in the memory 113, implements the automatic profile detection method provided in any of the foregoing method embodiments.

[0098] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0099] Therefore, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the automatic profile detection method provided in any of the foregoing method embodiments.

[0100] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0103] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0106] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can apply this application to the technical scope disclosed herein.

[0108] Within this scope, various equivalent modifications or replacements can be easily conceived, and these modifications or replacements should all be covered.

[0109] This application falls within the scope of protection of this application. Therefore, the scope of protection of this application should be defined by the claims.

[0110] The scope of protection shall prevail.

Claims

1. An automatic inspection method for profiles, characterized in that, include: The vehicle body is detected in the image of the loading material, the vehicle body area image is determined, and the vehicle body area image is input into the pre-trained profile detection model so that the profile detection model can perform profile detection on the vehicle body area image and output multiple profile prediction boxes. Based on the shape features of the prediction boxes, the multiple profile prediction boxes are filtered once to obtain a first set of filtered prediction boxes. Based on the position features of the prediction boxes, the first set of filtered prediction boxes is filtered a second time to obtain a second set of filtered prediction boxes. Based on the distance features of the prediction boxes, the second set of filtered prediction boxes is filtered a third time to obtain a third set of filtered prediction boxes. The third set of filtering prediction boxes is counted to obtain the number of profiles in the aluminum material image.

2. The method as described in claim 1, characterized in that, Based on the shape characteristics of the prediction boxes, the multiple profile prediction boxes are filtered once to obtain a first set of filtered prediction boxes, which specifically includes: Obtain the prediction frame shape features corresponding to each of the multiple profile prediction frames, wherein the prediction frame shape features include prediction frame length, prediction frame width, prediction frame area, and prediction frame aspect ratio; Determine whether the length of the prediction box is within a preset length range, whether the width of the prediction box is within a preset width range, whether the area of ​​the prediction box is within a preset area range, and whether the aspect ratio of the prediction box is within a preset aspect ratio range; If any of the predicted frame lengths are not within the preset length range, the predicted frame width is not within the preset width range, the predicted frame area is not within the preset area range, or the predicted frame aspect ratio is not within the preset aspect ratio range, then the profile predicted frame corresponding to the current predicted frame shape feature is removed. Based on the predicted box shape features, the mean of the predicted box shape features and the standard deviation of the predicted box shape features are calculated, and a normal distribution confidence interval is constructed based on the mean of the predicted box shape features and the standard deviation of the predicted box shape features. Determine whether the current prediction frame shape feature exceeds the normal distribution confidence interval. If so, remove the profile prediction frame corresponding to the current prediction frame shape feature. Integrate all the retained profile prediction frames to obtain the first set of filter prediction frames.

3. The method as described in claim 1, characterized in that, Based on the location features of the predicted bounding boxes, a second filtering process is performed on the first set of filtered predicted bounding boxes to obtain a second set of filtered predicted bounding boxes, specifically including: Obtain the coordinates of the first filtering prediction box corresponding to each first filtering prediction box in the first filtering prediction box set. Compare the coordinates of the first filtering prediction box with the boundary of the rectangle corresponding to the carriage area image. If the first filtering prediction box is outside the boundary of the rectangle, the current first filtering prediction box is removed. If the first filtering prediction box is inside the boundary of the rectangle, the current first filtering prediction box is retained. And obtain the first ordinate corresponding to each first filter prediction box in the first filter prediction box set. Based on the first ordinate, sort each first filter prediction box in descending order and perform layering processing to assign each first filter prediction box to a matching first filter prediction box layer. The first filtering prediction box in the first filtering prediction box set is traversed in a layer order from bottom to top to obtain multiple layers of prediction box counts. The count of each layer of prediction box is compared with a preset layer of prediction box count threshold. If the count of the layer of prediction box is less than the preset layer of prediction box count threshold, all first filtering prediction boxes in the first filtering prediction box layer corresponding to the count of the layer of prediction box count are removed. Integrate all the retained first-filter prediction boxes to obtain the second-filter prediction box set.

4. The method as described in claim 1, characterized in that, Based on the distance features of the predicted bounding boxes, the second set of filtered predicted bounding boxes is subjected to three filtering processes to obtain the third set of filtered predicted bounding boxes, which specifically includes: A density clustering algorithm based on rectangular distance is used to cluster the second set of filtering prediction boxes, so as to divide the second set of filtering prediction boxes into multiple clusters; Determine the largest cluster among the plurality of clusters, remove all second filter prediction boxes corresponding to the non-largest cluster among the plurality of clusters, and retain all second filter prediction boxes corresponding to the largest cluster; All the retained second-filter prediction boxes are integrated to obtain the third-filter prediction box set.

5. The method as described in claim 4, characterized in that, A density-based clustering algorithm based on rectangular distance is used to cluster the second set of filtered prediction boxes into multiple clusters, specifically including: Set a corresponding unvisited marker, distance range parameter, and neighbor point number threshold for each second filter prediction box in the second filter prediction box set; Iterate through each second filter prediction box in the second filter prediction box set in sequence, and change the unvisited mark of the current second filter prediction box to the visited mark; If the number of first neighbor prediction boxes within the distance range parameter corresponding to the current second filtering prediction box is not less than the threshold of the number of neighbor points, then the current second filtering prediction box is taken as the core point, and a first cluster is created based on the core point; Add all first neighborhood prediction boxes corresponding to the current second filtering prediction box to the queue to be processed; If the queue to be processed is not empty, select any first neighborhood prediction box from the queue to be processed as the target first neighborhood prediction box. If the marker corresponding to the target first neighborhood prediction box is an unvisited marker, then modify the unvisited marker of the target first neighborhood prediction box to the visited marker. Obtain all second neighbor prediction boxes corresponding to the target first neighbor prediction box. If the number of second neighbor prediction boxes corresponding to all second neighbor prediction boxes is not less than the threshold of the number of neighbor points, add all second neighbor prediction boxes to the queue to be processed. Check whether the target first neighbor prediction box has been assigned to a cluster. If not, assign the target first neighbor prediction box from the queue to the first cluster, and re-determine whether the queue to be processed is empty. If the processing queue is empty, the creation of the first cluster is completed; Check if every second filter prediction box in the second filter prediction box set has been traversed. If not, traverse every second filter prediction box in the second filter prediction box set again. Otherwise, integrate all first clusters to obtain multiple clusters.

6. The method as described in claim 1, characterized in that, Perform carriage inspection on the aluminum material image to determine the carriage area image, specifically including: The image of the aluminum material to be detected is input into a pre-trained carriage detection model, so that the carriage detection model can detect the carriage in the image of the aluminum material and output the carriage prediction box and the carriage confidence corresponding to the carriage prediction box. The confidence level of the carriage is compared with a preset confidence threshold for the carriage. If the confidence level of the carriage is greater than the preset confidence threshold for the carriage, the predicted carriage frame is retained, and the image region corresponding to the predicted carriage frame is used as the carriage region image. The pre-training process of the carriage detection model includes: Based on the edge detection algorithm, each loading material sample image in the collected loading material sample image set is detected, the candidate connected region of the cargo compartment in the loading material sample image is extracted, the minimum bounding rectangle of the candidate connected region of the cargo compartment is calculated, the first cargo compartment prediction box is determined, and the first cargo compartment prediction box is labeled to obtain the truck cargo compartment labeled image dataset. The truck body labeled image dataset is input into the YOLOv8 model, so that the YOLOv8 model extracts multi-scale features of the image through the backbone network for each truck body labeled image in the truck body labeled image dataset, performs feature fusion of the multi-scale features of the image through the neck network, and outputs the model prediction result by the head network. The model prediction result includes the coordinates of the truck body predicted box and the confidence score of the truck body predicted box corresponding to the coordinates of the truck body predicted box. Calculate the loss value between the model's predicted result and the actual result. Based on the loss value, use the backpropagation algorithm to perform backpropagation optimization on the model parameters of the YOLOv8 model to obtain the optimized carriage detection model.

7. The method as described in claim 1, characterized in that, The model training process of the profile inspection model specifically includes: Obtain sample image sets of regular carriage areas and sample image sets of abnormal carriage areas; Based on the sample image set of the conventional carriage area, the initial profile detection model is trained with a first learning rate to obtain the first profile detection model. Based on the sample image set of the abnormal carriage area, the first profile detection model is trained with a second learning rate to obtain the second profile detection model; The second profile detection model is trained using a third learning rate based on the sample image set of the normal carriage area and the sample image set of the abnormal carriage area to obtain the profile detection model.

8. An automatic profile inspection device, characterized in that, include: The module consists of a carriage detection module, a prediction box filtering module, and a counting module. The carriage detection module is used to perform carriage detection on the loading material image, determine the carriage area image, and input the carriage area image into the pre-trained profile detection model so that the profile detection model can perform profile detection on the carriage area image and output multiple profile prediction boxes. The prediction box filtering module is used to perform a first filtering process on the multiple profile prediction boxes based on the prediction box shape features to obtain a first set of filtered prediction boxes; to perform a second filtering process on the first set of filtered prediction boxes based on the prediction box position features to obtain a second set of filtered prediction boxes; and to perform a third filtering process on the second set of filtered prediction boxes based on the prediction box distance features to obtain a third set of filtered prediction boxes. The counting module is used to perform counting processing on the third set of filtering prediction boxes to obtain the number of profiles in the aluminum material image.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images