Method and apparatus for detecting an elongated object

Abstract

The application belongs to the technical field of artificial intelligence, and discloses a detection method and a detection device for an elongated target, the detection method comprising: acquiring a target image to be detected; extracting first feature information of the target image to be detected in different preset directions respectively; splicing the first feature information to obtain target feature information; and inputting the target feature information into a target detection network to output the category and position information of the elongated target in the target image to be detected. The application solves the problem that the existing detection scheme for defects cannot accurately detect elongated defects.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and particularly relates to a detection method and a detection device for an elongated target. BACKGROUND

[0002] In industrial production, defect detection is of great importance. Defect detection can discover and eliminate unqualified products early, ensuring that the performance of the final product meets the expected standard. Moreover, high-quality products obtained through defect detection can enhance customer trust and satisfaction, thereby strengthening brand image. In addition, by detecting and eliminating defective products in a timely manner, waste product rate can be significantly reduced, and material waste can be minimized. Furthermore, early detection of defects can avoid rework and maintenance in the later stage, thereby saving a large amount of time and money. Therefore, defect detection has always been one of the core problems that enterprises focus on.

[0003] Early defect detection mainly relies on manual inspection, but manual inspection is slow and difficult to meet the high-speed requirements of large-scale production. Moreover, different inspectors may have different judgment standards, resulting in inconsistent inspection results. In addition, long-term repetitive work can easily cause fatigue and increase the risk of missed detection. Obviously, this method is not only inefficient, but also susceptible to human factors, leading to missed detection and false detection. With the development of manufacturing industry, the requirements for product quality are becoming higher and higher, and the traditional manual inspection method has been unable to meet the needs of modern production. Therefore, simple machines are gradually used for defect detection in the prior art, such as optical comparators, ultrasonic detectors, and X-ray detectors. Although traditional mechanical detection equipment can improve efficiency to some extent, its detection precision and flexibility are still limited. Moreover, these devices can only detect specific types of defects and are difficult to identify complex or subtle defects.

[0004] In order to overcome the above limitations, the prior art has appeared a method of using machine vision for defect detection. The specific detection process is as follows: an industrial camera is used to capture an image of the object to be detected; the image is processed for denoising, enhancement, etc.; key features in the image are extracted; template matching and machine learning are used to detect defects; the detected defects are classified and marked in the image. There also appears a method of defect detection based on deep learning technology, especially convolutional neural networks, which can automatically learn and identify various types of defects based on a large amount of data. Moreover, the application of transfer learning and reinforcement learning makes the model training more efficient and the detection effect more stable.

[0005] However, elongated targets often appear in industrial detection, and the information in the longitudinal and lateral directions of the elongated targets is extremely unbalanced. Currently, traditional machine vision methods usually rely on manually designed features such as edges, contours, and textures. For elongated targets, due to their very fine edges, possibly non-obvious or incomplete contours, and very limited texture information, the features of elongated targets may not be prominent or difficult to capture. The aspect ratio of elongated targets in an image varies greatly, which makes it difficult for traditional geometric feature-based methods (i.e., assuming that the target has a standard structure or shape) to effectively handle it. For example, although Hough transform can be used to detect straight lines, it is not effective for detecting curved or irregular elongated targets.

[0006] Deep learning models can detect targets with complex shapes, but elongated targets (such as wires, pipes, and blood vessels) often have complex shapes and structures. These targets can appear in various poses in an image, such as bending, twisting, or partial occlusion. In order for the model to accurately identify these complex and variable shapes, a large amount of labeled data is needed to cover as many variations as possible. However, elongated targets may appear less frequently in images, i.e., the dataset of elongated targets is small, which can lead to an imbalance between positive and negative samples in the dataset, affecting the prediction accuracy of the model. And the small dataset of elongated targets is prone to overfitting, i.e., it performs well on training data but poorly on new data, so that when the model is applied to real-world situations, it cannot accurately predict elongated targets. Therefore, the various defect detection schemes in the prior art cannot accurately detect elongated defects.

[0007] In summary, there is an urgent need in the prior art for a detection scheme that can accurately detect elongated defects. SUMMARY

[0008] The present application provides a detection method and a detection device for elongated targets to solve the problem that the defect detection scheme in the prior art cannot accurately detect elongated defects.

[0009] To solve the above technical problems, in a first aspect, the present application provides a detection method for elongated targets, comprising:

[0010] obtaining a target image to be detected;

[0011] extracting first feature information of the target image to be detected in different preset directions, respectively;

[0012] splicing each of the first feature information to obtain target feature information;

[0013] inputting the target feature information into a target detection network to output the category and position information of the elongated target in the target image to be detected.

[0014] Optionally, the extracting the first feature information of the to-be-detected target image in different preset directions respectively comprises:

[0015] splitting the target convolutional layer into one-dimensional convolutional layers, wherein the number of the one-dimensional convolutional layers is same as the dimension of the target convolutional layer;

[0016] inputting the to-be-detected target image into each one-dimensional convolutional layer respectively to extract the first feature information of the to-be-detected target image in different preset directions.

[0017] Optionally, the splicing each first feature information to obtain target feature information comprises:

[0018] inputting the to-be-detected target image into the target convolutional layer to extract second feature information of the to-be-detected target image, wherein the number of channels of the to-be-detected target image input into the target convolutional layer is greater than the number of the target convolutional layer;

[0019] splicing the second feature information and the first feature information to obtain target feature information.

[0020] Optionally, the inputting the to-be-detected target image into each one-dimensional convolutional layer respectively to extract the first feature information of the to-be-detected target image in different preset directions comprises:

[0021] inputting the to-be-detected target image into a backbone feature extraction network in a target detection model to output third feature information;

[0022] inputting the third feature information into each one-dimensional convolutional layer in a neck feature extraction network in the target detection model to output the first feature information in different preset directions.

[0023] Optionally, the splicing each first feature information to obtain target feature information comprises:

[0024] extracting original feature information of the to-be-detected target image;

[0025] splicing each first feature information and the original feature information to obtain target feature information.

[0026] Optionally, before the acquiring the to-be-detected target image, the method further comprises:

[0027] acquiring a to-be-detected image, and dividing the to-be-detected image into different connected domains according to pixel values of the to-be-detected image, wherein each connected domain corresponds to a unique pixel threshold segment;

[0028] For a target pixel threshold segment, a pixel value at a preset position in the target pixel threshold segment is selected as a target pixel value, and pixel values in a target connected domain corresponding to the target pixel threshold segment are unified as the target pixel value, to obtain a first to-be-detected image;

[0029] Obtain contour information in the first to-be-detected image;

[0030] According to the contour information, obtain an elongated target in the first to-be-detected image;

[0031] Display the elongated target in the first to-be-detected image, to generate the to-be-detected target image.

[0032] Optionally, the dividing the to-be-detected image into different connected domains according to pixel values of the to-be-detected image comprises:

[0033] Dividing the pixel values of the to-be-detected image into different pixel threshold segments;

[0034] Extracting pixel threshold values at ends of each pixel threshold segment to form initial population individuals in a genetic algorithm;

[0035] Performing multiple crossover and mutation on the initial population individuals to obtain new population individuals;

[0036] Inputting each new population individual and the initial population individual into a preset global fitness function to obtain fitness values of the new population individual and the initial population individual;

[0037] Taking a population with a maximum fitness value as a target population;

[0038] Determining pixel threshold values corresponding to each individual in the target population, and dividing the to-be-detected image into different connected domains according to the pixel threshold values.

[0039] Optionally, the global fitness function is:

[0040] σ 2 (t)=p i1 (T)*(μ i1 (T)―μ t ) 2 +p i0 (T)*(μ i0 (T)―μ t ) 2 ;

[0041] Wherein, σ 2 (T) is a fitness value of a population individual, p i1 (T) is a global ratio of foreground pixels of the population individual, μ i1(T) is the foreground pixel mean value of each individual in the population, μ t is the global pixel mean value of the population, p i0 (T) is the background pixel ratio of the population individual, μ i0 (T) is the background pixel mean value of each individual in the population, T is the pixel threshold value corresponding to the population individual.

[0042] Optionally, the method further comprises:

[0043] obtaining size information of the elongated target;

[0044] screening a target contour corresponding to the elongated target from the contour information according to the size information;

[0045] solving a convex hull contour corresponding to the target contour according to a convex hull algorithm;

[0046] taking the convex hull contour as the contour of the elongated target to obtain the elongated target in the first to-be-detected image.

[0047] In a second aspect, the present application provides a detection device for an elongated target, comprising:

[0048] an acquisition module configured to acquire a to-be-detected target image;

[0049] an extraction module configured to extract first feature information of the to-be-detected target image in different preset directions respectively;

[0050] a splicing module configured to splice each of the first feature information to obtain target feature information;

[0051] an output module configured to input the target feature information into a target detection network to output category and position information of an elongated target in the to-be-detected target image.

[0052] Compared with the prior art, the detection method for an elongated target provided by the present application has the following beneficial effects:

[0053] The scheme provided by the present application can extract the first feature information of the to-be-detected target image in different preset directions when extracting the feature information of the to-be-detected target image, so that the feature information of the to-be-detected target image in different preset directions can be deeply extracted, and the feature information of the to-be-detected target image in a certain direction will not be missed. Thus, compared with the prior art, the scheme provided by the present application avoids the problem that the features of the elongated target in the traditional machine vision method are difficult to capture and effectively process; and compared with the deep learning model in the prior art, the scheme provided by the present application accurately extracts the feature information of the to-be-detected target image in different preset directions, so that the target detection network can accurately predict the elongated target based on the accurate feature information, avoiding the problem that the deep learning model in the prior art cannot accurately predict the elongated target due to insufficient training data. In summary, the scheme provided by the present application solves the problem that the detection schemes for the elongated defect in the prior art cannot accurately detect the elongated defect. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical schemes in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some of the embodiments of the present application, not all the embodiments. For those skilled in the art, other drawings obtained according to these drawings without creative labor also belong to the scope of protection of the present application.

[0055] Figure 1 is a flowchart of an elongated target detection method provided by an embodiment of the present application.

[0056] Figure 2 is a flowchart of another elongated target detection method provided by an embodiment of the present application.

[0057] Figure 3 is a structural schematic diagram of a neural network structure provided by an embodiment of the present application.

[0058] Figure 4 is a structural schematic diagram of another neural network structure provided by an embodiment of the present application.

[0059] Figure 5 is a structural schematic diagram of another neural network structure provided by an embodiment of the present application.

[0060] Figure 6 is a flowchart of another elongated target detection method provided by an embodiment of the present application.

[0061] Figure 7It is an mAP50 image of a YOLOv8 model provided by an embodiment of the application.

[0062] Figure 8 It is an mAP50 image of an improved model provided by an embodiment of the application.

[0063] Figure 9 It is a confusion matrix graph of a YOLOv8 type provided by an embodiment of the application.

[0064] Figure 10 It is a confusion matrix graph of an improved model provided by an embodiment of the application.

[0065] Figure 11 It is a test result schematic diagram of an improved model provided by an embodiment of the application.

[0066] Figure 12 It is a recognition graph of an elongated target workpiece provided by an embodiment of the application.

[0067] Figure 13 It is a structural schematic diagram of an elongated target detection device provided by an embodiment of the application. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be given below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0069] In order to make the description of the present disclosure more detailed and complete, the following describes the embodiments of the present application; but this is not the only form of implementation or use of the specific embodiments of the present application. The embodiments include the features of the specific embodiments and the method steps and their order used to construct and operate these specific embodiments. However, other specific embodiments can also be used to achieve the same or equivalent functions and step sequences. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein.

[0071] In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two, and other quantifiers similar thereto should be understood as the preferred embodiments described herein are used only to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0072] As shown in Figure 1 A flow chart of a method for detecting an elongated target provided by the present application is shown, comprising the following steps.

[0073] Step S110, acquiring a target image to be detected.

[0074] It should be noted that the target image to be detected acquired in this step is an image containing the elongated target to be detected. The acquisition method of the target image to be detected can be any method that can be implemented, for example, an industrial camera can be used to take a picture or video of the elongated target to acquire the target image to be detected.

[0075] Step S120, extracting first feature information of the target image to be detected in different preset directions respectively.

[0076] It should be noted that in this step, the different preset directions can be any direction, which can be set according to the specific needs of the application, for example, a coordinate system can be established for the target image to be detected, and the different preset directions can be 30 degrees, 60 degrees, 90 degrees, 120 degrees, 150 degrees and 180 degrees in the coordinate system, and the above angles are the included angle between the preset direction and the positive half of the X axis. As a specific example, the different preset directions in this step can be the vertical direction and the horizontal direction, and further the feature information of the target image to be detected in the vertical direction and the horizontal direction can be extracted.

[0077] It should be noted that the way of extracting the first feature information of the target image to be detected in different preset directions in this step can be any method that can be implemented. For example, edge detection algorithm can be used to extract edge information in the image, and further analyze the distribution of these edges in the vertical and horizontal directions; the histogram of the image in different directions can also be calculated, and then detailed pixel distribution information can be obtained, and then the feature information of the image in the vertical and horizontal directions can be obtained; wavelet transform can also be used to extract the feature information of the image in the vertical and horizontal directions, etc.

[0078] Step S130, splicing each of the first feature information to obtain target feature information.

[0079] It should be noted that in this step, the manner of splicing each first feature information can be any manner that can be implemented, for example, the first feature information in different preset directions is spliced directly in the channel dimension, or a corresponding weight can be set for the first feature information in different preset directions, and the first feature information in each direction is weighted and fused according to different weights, or a convolutional neural network can be used to automatically learn how to fuse the first feature information in different preset directions.

[0080] In step S140, the target feature information is input into a target detection network, and the class and position information of the elongated target in the to-be-detected target image are output.

[0081] It should be noted that the target detection network in this step can be any target detection network that can output the class and position information of the elongated target in the to-be-detected target image after inputting the target feature information into the target detection network. For example, the target detection network can be a YOLO series model, a Single Shot MultiBox Detector (SSD) model, or a Deformable Convolutional Networks (DCN) model, etc. If the target feature information is input into the YOLO series model, the YOLO model will divide it into a grid, and perform boundary box prediction in each grid unit, and then the position information and class probability of the boundary box can be output, so as to determine the class and position information of the elongated target. If the target feature information is input into the SSD model, the target feature information can be passed to the convolutional layers of different scales of the SSD model, and on the feature map formed by each scale of the convolutional layer, a series of default boxes will be generated, and the offset and class of these default boxes will be predicted, and redundant boundary boxes are removed through non-maximum suppression, and finally the position information and class of the elongated target can be output. If the target feature information is input into the DCN model, the deformable convolutional layer of the DCN model can be used to further extract feature information, which can be used for classification and boundary box regression, and then the position information and class of the elongated target can be output.

[0082] It should be noted that in this step, the class of the elongated target can be obtained through the class of the elongated target finally output.

[0083] In this way, the scheme of the embodiment of the present application can extract the first feature information of the to-be-detected target image in different preset directions when extracting the feature information of the to-be-detected target image, so that the feature information of the to-be-detected target image in different preset directions can be extracted in depth, and the feature information of the to-be-detected target image in a certain direction will not be missed. In this way, compared with the prior art, the scheme of the embodiment of the present application avoids the problem that the features of the elongated target are difficult to capture and effectively process in the traditional machine vision method. Compared with the deep learning model in the prior art, the scheme of the embodiment of the present application accurately extracts the feature information of the to-be-detected target image in different preset directions, so that the target detection network can accurately predict the elongated target based on the accurate feature information, and the problem that the deep learning model cannot accurately predict the elongated target due to insufficient training data when detecting the elongated target in the prior art is avoided. In summary, the scheme of the embodiment of the present application solves the problem that the detection schemes of the elongated defects in the prior art cannot accurately detect the elongated defects.

[0084] In an alternative implementation, as shown in FIG. 12, step S120 comprises: Figure 2 As shown in FIG. 13, which is another flowchart of the embodiment of the present application, step S120 comprises:

[0085] Step S121 splits the target convolutional layer into one-dimensional convolutional layers.

[0086] Step S122 inputs the to-be-detected target image into each one-dimensional convolutional layer respectively, and extracts the first feature information of the to-be-detected target image in different preset directions.

[0087] The number of the one-dimensional convolutional layers is the same as the dimension of the target convolutional layer.

[0088] It should be noted that the dimension of the target convolutional layer can be set according to specific needs in application. For example, it can be a two-dimensional convolutional layer or a three-dimensional convolutional layer. Naturally, when the target convolutional layer is a two-dimensional convolutional layer, the number of the one-dimensional convolutional layers obtained by splitting the target convolutional layer is two, and when the target convolutional layer is a three-dimensional convolutional layer, the number of the one-dimensional convolutional layers obtained by splitting the target convolutional layer is three.

[0089] The height and length of the convolution kernel of the target convolutional layer can be set according to specific needs in application, for example, the height and length of the convolution kernel of the target convolutional layer are 3*3 or 5*5. Naturally, when the height and length of the convolution kernel of the target convolutional layer are 3*3, the height and length of the convolution kernel of the two one-dimensional convolutional layers obtained by splitting the target convolutional layer are 1*3 and 3*1 respectively, and when the height and length of the convolution kernel of the target convolutional layer are 5*5, the height and length of the convolution kernel of the two one-dimensional convolutional layers obtained by splitting the target convolutional layer are 1*5 and 5*1 respectively.

[0090] In this way, when the to-be-detected target image is input into each one-dimensional convolution layer, the first feature information of the to-be-detected target image in different preset directions can be extracted. For example, when the to-be-detected target image is input into the two one-dimensional convolution layers of 1*3 and 3*1 respectively split from the 3*3 target convolution layer, the first feature information of the to-be-detected target image in the vertical direction and the first feature information of the to-be-detected target image in the horizontal direction can be extracted respectively. For an elongated target, the 3*3 convolution kernel considers the information of a 3*3 region when extracting features. After being split into 1*3 and 3*1 convolution kernels, the 1*3 convolution kernel can better capture the feature changes of the target in the length direction (such as the long axis direction of the elongated target), and the 3*1 convolution kernel is better at capturing the feature changes of the target in the width direction (the short axis direction of the elongated target).

[0091] In addition, the split convolution kernels can have different parameter settings. For example, they can have different step sizes, padding modes, and the like. Such flexibility enables the model to better adapt to different shapes and sizes of elongated targets. In addition, during the calculation process, the calculation complexity can be reduced due to the smaller size of the convolution kernel. This is very advantageous for processing large-scale image data or running the model on resource-limited devices.

[0092] In an optional implementation, the step S130 comprises:

[0093] inputting the to-be-detected target image into the target convolution layer to extract second feature information of the to-be-detected target image;

[0094] splicing the second feature information and the first feature information to obtain target feature information.

[0095] In the implementation, the number of input channels of the to-be-detected target image into the target convolution layer is greater than the number of the target convolution layers.

[0096] It should be noted that the splicing manner of the second feature information and the first feature information can be any implementable manner.

[0097] It can be understood that the to-be-detected target image is input into the target convolution layer, and the number of input channels of the to-be-detected target image is greater than the number of the target convolution layers. Therefore, the original feature channels of the to-be-detected target image can be compressed into fewer channels. For an elongated target, numerous original feature channels can contain a large amount of local details, noise, and redundant information. The process of compressing channels is like removing these less important details, so that more representative semantic features can be extracted.

[0098] Optionally, in the present embodiment, the original feature information of the target image to be detected can also be extracted, and the original feature information is spliced with the first feature information and the second feature information to obtain target feature information, so that the target feature information used for target detection finally retains the original feature information of the elongated target. Retaining the original feature information of the elongated target can make the feature information retain the original information characteristics of the image, so that the obtained target feature information can retain the direct characteristics of the image, and the detection result of the elongated target based on the target feature information is more accurate. In addition, when the target image to be detected is input into the target convolutional layer, the compression of the channel will cause the target image to be detected to lose some information, and therefore splicing the original feature information can supplement the information lost by the second feature information and the first feature information.

[0099] As a specific example, as shown in FIG. 1, Figure 3 FIG. 2 shows an architecture diagram of a neural network provided by an embodiment of the present application, in which Figure 3 the number of target convolutional layers is assumed to be two, the original feature information of the target image to be detected is sequentially input into the two convolutional layers to obtain the second feature information, and the second feature information is spliced with the original feature information to output the final feature information. Such a neural network architecture is named as a first feature module in Figure 3 .

[0100] In an optional implementation, step S122 comprises:

[0101] inputting the target image to be detected into a backbone feature extraction network in the target detection model to output third feature information;

[0102] inputting the third feature information into each one-dimensional convolutional layer in a neck feature extraction network in the target detection model to output first feature information in different preset directions.

[0103] It should be noted that, in the present embodiment, the backbone feature extraction network extracts different levels of features of the target image to be detected through multi-layer convolutional operations. For example, the backbone feature extraction network can be a residual network, a visual geometry group network (VGG), or a mobile network convolutional neural network architecture. It can be understood that inputting the target image to be detected into the backbone feature extraction network can extract rich features from the target image to be detected.

[0104] It should be noted that in the present embodiment, the neck feature extraction network is located between the backbone feature extraction network and the detection head network in the target detection model, and is used to further process the features of the backbone feature extraction network to improve the detection performance. The neck feature extraction network can fuse feature information of different levels, enhance feature representation, and also enhance the quality of feature information. Specifically, the neck feature extraction network can be a feature pyramid network (FPN) or a path aggregation network (PANet), etc.

[0105] It can be understood that in the present embodiment, the one-dimensional convolutional layer obtained by splitting the target convolutional layer is arranged in the neck feature extraction network, which can utilize the features preliminarily extracted by the backbone, finely process the features, and then provide more suitable slender target features to the detection head, balance the abstract degree of the features, and will not interfere with the backbone network structure and the general feature extraction process, which is conducive to balancing the feature extraction of multiple target types, reducing the burden of the detection head, improving the efficiency and accuracy of slender target detection, and better cooperating with other modules for feature fusion. However, the detection head network is not conducive to exerting its fusion advantage.

[0106] In an optional implementation, the step S130 comprises:

[0107] extracting original feature information of the target image to be detected;

[0108] splicing each of the first feature information and the original feature information to obtain target feature information.

[0109] It should be noted that in the present embodiment, the original feature information of the target to be detected is the feature information directly extracted from the target image to be detected, which has not been further processed or enhanced.

[0110] It can be understood that the original feature information and the first feature information are spliced together to obtain the target feature information, so that the target feature information used for target detection finally retains the original feature information of the slender target. Retaining the original feature information of the slender target can make the feature information retain the original information characteristics of the image, so that the obtained target feature information can retain the direct characteristics of the image, and the detection result of the slender target based on the target feature information is more accurate. In addition, when the target image to be detected is input into the target convolutional layer, the compression of the channel will cause the target image to be detected to lose some information, and therefore splicing the original feature information can supplement the information lost by the first feature information.

[0111] As a specific example, as shown in FIG. 6, the original feature information of the target to be detected is obtained by directly extracting the target image to be detected, and the first feature information is obtained by inputting the target image to be detected into the target convolutional layer. The original feature information and the first feature information are spliced together to obtain the target feature information. Figure 4Fig. 2 shows an architecture diagram of another neural network provided by an embodiment of the present application, in which Figure 4 In the embodiment, after the original feature information is input into two one-dimensional convolutional layers respectively to obtain two first feature information for splicing, the first feature information can be spliced with the original feature information to obtain the final output feature information. Such a neural network architecture is named as a second feature module in the embodiment. Figure 4

[0112] As a specific example, as shown in Fig. 3, a neural network architecture provided by an embodiment of the present application is shown, in which Figure 5 Fig. 4 shows an architecture diagram of another neural network provided by an embodiment of the present application, in which Figure 5 In the embodiment, the feature information of the target image to be detected can be input into a convolutional layer as input to obtain original feature information. Then, the original feature information can be input into a first feature module and a second feature module respectively. The feature information output by the two feature modules can be spliced and input into a normalization module for normalization to obtain target feature information. The target feature information is input into an efficient channel attention mechanism module (i.e., a convolutional block attention module) to output first target feature information. The first target feature information can be spliced with the original feature information to obtain the final output feature information. Figure 5 The module shown in Fig. 5 can be called a target feature module. Specifically, the algorithm executed by the normalization module can be shown in the following formula:

[0113]

[0114] wherein γ is a preset scaling parameter, β is a preset offset parameter, ∈ is a preset constant with a very small value, which ensures that the denominator is not 0, E(x) is the expectation of x, σ(x) is the variance of x, x is the feature information input into the normalization module, and y is the feature information output by the normalization module.

[0115] Optionally, in the embodiment, the feature information of the original feature information in different preset directions can be extracted to obtain first feature information in different preset directions. The first feature information in different preset directions and the original feature information can be spliced to obtain target feature information.

[0116] In an optional embodiment, as shown in Fig. 6, a flowchart of another method for detecting an elongated target provided by an embodiment of the present application is shown, which further includes the following steps before step S110: Figure 6

[0117] In step S101, an image to be detected is acquired, and the image to be detected is divided into different connected domains according to pixel values of the image to be detected.

[0118] Each connected domain corresponds to a unique pixel threshold segment.

[0119] ​​It should be noted that in this step, the image to be detected is an image with an elongated target. It can be understood that the image to be detected has a corresponding pixel value range, that is, a pixel threshold range between the minimum value and the maximum value of all pixels in the image to be detected. In this step, "dividing the image to be detected into different connected domains according to the pixel value of the image to be detected" means "dividing the pixel threshold range corresponding to the image to be detected into a plurality of continuous pixel threshold segments, dividing the image to be detected corresponding to each pixel threshold segment to form a connected domain".

[0120] Optionally, since the color image has three pixel channels, namely red, green and blue channels, if the connected domain division is directly performed on the color image to be detected based on the pixel value, each pixel channel needs to be divided, and therefore, in this step, the color image to be detected can be converted into a gray image format first. The image corresponding to the gray image format has only one pixel channel, so that the division resources of the image to be detected can be effectively saved.

[0121] In step S102, for a target pixel threshold segment, a pixel value at a preset position in the target pixel threshold segment is selected as a target pixel value, and the pixel values in a target connected domain corresponding to the target pixel threshold segment are uniformly set to the target pixel value, to obtain a first image to be detected.

[0122] It should be noted that in this step, the target pixel threshold segment is any one of the pixel threshold segments corresponding to the image to be detected.

[0123] It should be noted that the preset position in the target pixel threshold segment can be set according to specific needs in application. For example, in order to make the first image to be detected obtained in this step clearer and easier to identify, the maximum pixel value in the target pixel threshold segment can be selected as the target pixel value, and the pixel values in the target connected domain are uniformly set to the maximum pixel value.

[0124] It should be noted that the manner of uniformly setting the pixel values in the target connected domain to the target pixel value in this step can be any manner that can be implemented, for example, all the pixel values in the target connected domain can be directly set to the target pixel value, or the connected domain can be processed through inflation and corrosion, and then the target pixel value is assigned to the target connected domain.

[0125] It can be understood that uniformly setting the pixel values in the target connected domain to the target pixel value in this step can enhance the contrast between the target connected domain and other connected domains, so as to highlight the boundary in the image to be detected, so that the outline of the elongated target in the image to be detected is highlighted.

[0126] In step S103, the outline information in the first image to be detected is obtained.

[0127] Step S104, obtaining the elongated target in the first to-be-detected image according to the contour information.

[0128] It can be understood that through the above steps, various boundaries (i.e. the contour information in this step) in the to-be-detected image can be revealed, and the contour of the elongated target is among these boundaries, and then the contour of the elongated target in the first to-be-detected image can be obtained among these boundaries to determine the elongated target in the first to-be-detected image.

[0129] Step S105, displaying the elongated target in the first to-be-detected image to generate the to-be-detected target image.

[0130] It should be noted that in this step, the way of displaying the elongated target in the first to-be-detected image can be any implementable way. For example, for the first to-be-detected image after grayscale processing, the inside of the contour information of the elongated target in the first to-be-detected image can be filled with (255, 255, 255) pixels, converted into binary data (1, 1, 1), and the outside background of the contour information is filled with pixel binary data (0, 0, 0), and finally the first to-be-detected image after pixel filling is ANDed with the colored to-be-detected image. In this way, the pixels of the elongated target in the colored to-be-detected image remain unchanged, and since (0, 0, 0) is black, the background outside the elongated target is filled with a black background. In this way, the finally generated to-be-detected target image contains a colored elongated target and a black background outside the elongated target. It can be understood that the black background can make the colored elongated target more conspicuous.

[0131] It can be understood that the elongated target is often easily confused with other lines or structures in the background, which increases the difficulty of detection. Steps S101-S105 can effectively extract the elongated target to facilitate the target detection network to accurately output the position information and the category of the elongated target.

[0132] In an optional implementation, step S101 includes:

[0133] dividing the pixel values of the to-be-detected image into different pixel threshold segments;

[0134] extracting the pixel threshold values at the ends of each of the pixel threshold segments to form initial population individuals in the genetic algorithm;

[0135] performing multiple crossover and mutation on the initial population individuals to obtain new population individuals;

[0136] input the new population individuals and the initial population individuals into a preset global fitness function to obtain fitness values of the new population individuals and the initial population individuals;

[0137] take the population with the maximum fitness value as a target population;

[0138] determine pixel threshold values corresponding to each individual in the target population, and divide the to-be-detected image into different connected domains according to the pixel threshold values.

[0139] It can be understood that dividing pixel values of the to-be-detected image into different pixel threshold segments means that, in all pixel values of the to-be-detected image, a plurality of pixel values are extracted as pixel threshold values according to an arrangement sequence from small to large, and a pixel threshold segment is naturally formed between two adjacent pixel threshold values. Further, in the implementation mode, the plurality of extracted pixel values are set as individuals of a population, and these individuals can form an initial population.

[0140] After obtaining the initial population, the initial population can be initialized, and the number of iterations, the crossover probability, and the mutation probability of population initialization can be set according to specific needs in application, for example, the number of iterations can be set to 100, the crossover probability can be set to 0.2, and the mutation probability can be set to 0.06. Further, after initializing the initial population, the population can be crossed and mutated based on the above-mentioned crossover probability and mutation probability, and the individuals that are specifically crossed and mutated can be set according to needs in application, which is not limited in detail here. After the population is finally crossed and mutated, a plurality of new populations can be obtained by continuously iterating based on the number of iterations (that is, a new population is generated after the population is crossed and mutated once, and the next new population is obtained after the previous new population is crossed and mutated).

[0141] It should be noted that in the implementation mode, the global fitness function can be set according to specific needs in application.

[0142] It can be understood that the target population finally obtained is the population with the maximum fitness value, that is, each individual in the target population is the best pixel threshold value for highlighting the boundary in the to-be-detected image that needs to be found. In this way, the connected domains are divided according to these best pixel threshold values, and then the pixels in the connected domains are unified as a set threshold value (for example, the maximum pixel threshold value in the pixel threshold segment), which can better highlight the contour information in the to-be-detected image.

[0143] In an optional implementation mode, the global fitness function is:

[0144] σ 2 (T)=p i1 (T)*(μ i1 (T)―μt 2 p i0 (T)*(μ i0 (T)―μ t ) 2

[0145] wherein, σ 2 (T) is the fitness value of the population individual, p i1 (T) is the foreground pixel ratio of the population individual, μ i1 (T) is the foreground pixel mean of each individual in the population, μ t is the global pixel mean of the population, p i0 (T) is the background pixel ratio of the population individual, μ i0 (T) is the background pixel mean of each individual in the population, and T is the pixel threshold value corresponding to the population individual.

[0146] It should be noted that the global fitness function in the present implementation is the calculation formula of the maximum inter-class variance. For detailed explanations of the parameters, refer to the parameter definitions of the maximum inter-class variance.

[0147] It can be understood that p i1 (T)*(μ i1 (T)―μ t ) 2 can represent the maximum value of the foreground pixel distance of the population individual from the global pixel mean, p i0 (T)*(μ i0 (T)―μ t ) 2 can represent the maximum value of the background pixel distance of the population individual from the global pixel mean. Since the global pixel mean is a constant value, when σ 2 (T) takes the maximum value, p i1 (T)*(μ i1 (T)―μ t ) 2 and p i0 (T)*(μ i0 (T)―μ t ) 2 will all take the maximum value. Naturally, the values of μ i1 (T) and μ i0 (T) are farthest apart at this time. Naturally, the foreground and background boundaries of the corresponding pixel threshold value are clearer at this time. Therefore, the corresponding pixel threshold value at this time is the optimal solution, and the outline formed based on the population threshold value is naturally also very clear.

[0148] In an alternative implementation, step S104 comprises:

[0149] obtaining the size information of the elongated target;​​

[0150] According to the size information, target contours corresponding to the elongated target are screened from the contour information;

[0151] According to a convex hull algorithm, a convex hull contour corresponding to the target contour is solved;

[0152] The convex hull contour is taken as the contour of the elongated target, and the elongated target in the first to-be-detected image is obtained.

[0153] It should be noted that the size information of the elongated target in the present implementation mode is the actual size information of the elongated target, which can be measured on site or obtained by pre-measurement.

[0154] Further, the size information of each contour information in the first to-be-detected image is measured, the size information of each contour information corresponding to the size information of the elongated target is selected, and the contour information is determined as the contour information of the elongated target.

[0155] Specifically, the convex hull algorithm in the present implementation mode can be any convex hull algorithm. The convex hull algorithm is an algorithm for finding the smallest convex polygon of a set of points (the smallest convex polygon containing all points). Obviously, the set of points of the convex hull algorithm mentioned in the present implementation mode is the point corresponding to the contour information of the elongated target. It can be understood that the convex hull algorithm can accurately find the smallest convex polygon containing all given points, which can accurately extract the target contour of the elongated target. Moreover, the contour generated by the convex hull algorithm is a closed polygon, which ensures the integrity of the contour.

[0156] As a specific example, after the detection method of the elongated target described above is adopted in the embodiment of the present application, as shown in Figure 7 and Figure 8 the mAP50 images of target detection obtained by using the YOLOv8 model and the target detection model (improved model) of the present application are shown, respectively. mAP is the mean average precision, which is an important indicator for measuring the performance of the model in target detection. It comprehensively considers the detection accuracy of the model for different categories and the importance of each category. The "50" in mAP50 refers to the intersection over union threshold of 0.5. The intersection over union is the ratio of the intersection area of the predicted bounding box and the true bounding box to the union area, which is used to measure the accuracy of the prediction.

[0157] As a specific example, after the detection method of the elongated target described above is adopted in the embodiment of the present application, as shown in Figure 9 and Figure 10The confusion matrix diagram of target detection obtained by using the YOLOv8 model and the confusion matrix diagram of target detection obtained by using the target detection model (improved model) of the application are shown.

[0158] As a specific example, point dispensing defects (one kind of elongated defects) are used as the data set to collect 1970 point dispensing defect pictures, some pictures may have multiple defects, and the data set is divided into a training set, a test set and a validation set according to a ratio of 8:1:1. The original model (using the YOLOv8n model) and the improved model (using the target detection method of the application) are trained on the pictures in the training set, and the training weights are imported into the network structure to test the test set. The specific test results are as shown in the table. Figure 11

[0159] As a specific example, as shown in Figure 12 The recognition diagram of a certain elongated target workpiece detected by using the target detection method of the application is shown.

[0160] Based on the above-mentioned elongated target detection method, the embodiment of the application provides an elongated target detection device, as shown in Figure 13 The elongated target detection device comprises:

[0161] The acquisition module 1310 is configured to acquire a target image to be detected;

[0162] The extraction module 1320 is configured to extract first feature information of the target image to be detected in different preset directions respectively;

[0163] The splicing module 1330 is configured to splice each of the first feature information to obtain target feature information;

[0164] The output module 1340 is configured to input the target feature information into a target detection network, and output the type and position information of the elongated target in the target image to be detected.

[0165] For other details of the implementation of each module in the above-mentioned elongated target detection device to achieve the above-mentioned technical solutions, refer to the description of the above-mentioned elongated target detection method provided in the embodiment of the application, which will not be repeated here.

[0166] ​In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the modules is merely a logical function division. There can be another division manner for the actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or modules, and can be in electrical, mechanical or other forms.

[0167] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed to a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0168] In addition, each function module in each embodiment of the present application can be integrated in a processing module, or each module can be physically present alone, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. If the integrated module is realized in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0169] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product.

[0170] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that the computer can store or be integrated into a server, data center and other data storage devices containing one or more available media sets. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media, or semiconductor media (such as solid state disk (SSD)) and the like.

[0171] The above describes the technical solutions provided by the present application in detail. The principles and implementation modes of the present application are described by applying specific examples. The above examples are only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; therefore, the content of the specification should not be understood as a limitation of the present application.

[0172] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer usable program code.

[0173] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device realize the functions described in the flowchart and / or block diagram. Figure 1 one flow or multiple flows and / or blocksFigure 1 means for performing the function specified in the block or blocks.

[0174] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a Figure 1 one or more flows and / or blocks Figure 1 means for performing the function specified in the block or blocks.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 steps of means for performing the function specified in the block or blocks.

[0176] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the apparent, such modifications and variations are to be included within the scope of the application and the equivalent of such.

Claims

1. A method of detecting an elongated object, characterized by, The method comprises: acquiring a target image to be detected; before acquiring the target image to be detected, further comprising: acquiring an image to be detected, and dividing the image to be detected into different connected domains according to pixel values of the image to be detected, wherein each connected domain corresponds to a unique pixel threshold segment; for a target pixel threshold segment, selecting a pixel value at a preset position in the target pixel threshold segment as a target pixel value, and unifying pixel values in a target connected domain corresponding to the target pixel threshold segment to the target pixel value, to obtain a first image to be detected; acquiring contour information in the first image to be detected; according to the contour information, obtaining an elongated target in the first image to be detected; displaying the elongated target in the first image to be detected, to generate the target image to be detected; the dividing the image to be detected into different connected domains according to pixel values of the image to be detected comprises: dividing pixel values of the image to be detected into different pixel threshold segments; extracting pixel threshold values at ends of each pixel threshold segment to form initial population individuals in a genetic algorithm; performing multiple crossover and mutation on the initial population individuals to obtain new population individuals; inputting each new population individual and the initial population individual into a preset global fitness function to obtain fitness values of the new population individual and the initial population individual; taking a population with a maximum fitness value as a target population; determining pixel threshold values corresponding to each individual in the target population, and dividing the image to be detected into different connected domains according to the pixel threshold values; extracting first feature information of the target image to be detected in different preset directions respectively; the extracting first feature information of the target image to be detected in different preset directions respectively comprises: splitting a target convolutional layer into one-dimensional convolutional layers, wherein the number of the one-dimensional convolutional layers is the same as the dimension of the target convolutional layer; inputting the target image to be detected into each one-dimensional convolutional layer to extract first feature information of the target image to be detected in different preset directions; splicing each first feature information to obtain target feature information; inputting the target feature information into a target detection network to output category and position information of an elongated target in the target image to be detected.

2. The method of detecting an elongated object according to claim 1, wherein the splicing each first feature information to obtain target feature information comprises: inputting the target image to be detected into the target convolutional layer to extract second feature information of the target image to be detected, wherein the number of channels of the target image to be detected input into the target convolutional layer is greater than the number of the target convolutional layers; splicing the second feature information and the first feature information to obtain target feature information.

3. The method of detecting an elongated object according to claim 1, wherein the inputting the target image to be detected into each one-dimensional convolutional layer to extract first feature information of the target image to be detected in different preset directions comprises: inputting the target image to be detected into a backbone feature extraction network in a target detection model to output third feature information; The third feature information is input into each one-dimensional convolutional layer in a neck feature extraction network in the target detection model, and first feature information in different preset directions is output.

4. The method of detecting an elongated object according to claim 1, wherein The first feature information is spliced to obtain target feature information, including: Extracting original feature information of the target image to be detected; Splicing each first feature information and the original feature information to obtain target feature information.

5. The method of detecting an elongated object according to claim 1, wherein The global fitness function is: σ 2 (T) = p i1 (T)*(μ i1 (T) - μ t ) 2 +p i0 (T)*(μ i0 (T) - μ t ) 2 ; wherein σ 2 (T) is the fitness value of the population individual, p i1 (T) is the foreground pixel ratio of the population individual, μ i1 (T) is the foreground pixel mean of each individual in the population, μ t is the global pixel mean of the population, p i0 (T) is the background pixel ratio of the population individual, μ i0 (T) is the background pixel mean of each individual in the population, T is the pixel threshold value corresponding to the population individual.

6. The method of detecting an elongated object according to claim 1, wherein According to the contour information, the elongated target in the first target image to be detected includes: Obtaining size information of the elongated target; According to the size information, the target contour corresponding to the elongated target is screened out from the contour information; According to the convex hull algorithm, the convex hull contour corresponding to the target contour is solved; The convex hull contour is taken as the contour of the elongated target to obtain the elongated target in the first target image to be detected.

7. An apparatus for detecting an elongated object, the apparatus being used to implement the steps of the method for detecting an elongated object according to any one of claims 1 to 6, characterized in that, Including: An acquisition module is configured to acquire a target image to be detected; An extraction module is configured to extract first feature information of the target image to be detected in different preset directions respectively; A splicing module is configured to splice each first feature information to obtain target feature information; An output module is configured to input the target feature information into a target detection network to output category and position information of an elongated target in the target image to be detected.

Images